You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2015/07/30 02:38:29 UTC

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 914 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/914/

All tests passed

Build Log:
[...truncated 12024 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150729_173002_666.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: PermGen space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid7516.hprof ...
   [junit4] Heap dump file created [126167231 bytes in 1.171 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150729_173002_666.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "updateExecutor-3496-thread-7"
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RMI RenewClean-[127.0.0.1:52564]"
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "SUITE-TestCloudSchemaless-seed#[89DC86373E70253A]"
   [junit4] java.lang.OutOfMemoryError: PermGen space
   [junit4] java.lang.OutOfMemoryError: PermGen space
   [junit4] <<< JVM J2: EOF ----

[...truncated 91 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150729_173002_666.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid7517.hprof ...
   [junit4] Heap dump file created [623820956 bytes in 6.115 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150729_173002_666.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "collection3"
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: Java heap space
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=89DC86373E70253A -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.3.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=5.3.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/test-framework/lib/junit4-ant-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/common/lucene-analyzers-common-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/lucene-codecs-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/lucene-backward-codecs-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/highlighter/lucene-highlighter-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/memory/lucene-memory-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/lucene-misc-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/lucene-spatial-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/expressions/lucene-expressions-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/suggest/lucene-suggest-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/grouping/lucene-grouping-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queries/lucene-queries-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/lucene-queryparser-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/lucene-join-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/lucene-sandbox-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/antlr-runtime-3.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-commons-4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/concurrentlinkedhashmap-lru-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hppc-0.5.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/spatial4j-0.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150729_173002_666.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150729_173002_666.suites
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)
   [junit4] ERROR: JVM J2 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=89DC86373E70253A -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.3.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=5.3.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/test-framework/lib/junit4-ant-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/common/lucene-analyzers-common-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/lucene-codecs-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/lucene-backward-codecs-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/highlighter/lucene-highlighter-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/memory/lucene-memory-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/lucene-misc-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/lucene-spatial-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/expressions/lucene-expressions-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/suggest/lucene-suggest-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/grouping/lucene-grouping-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queries/lucene-queries-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/lucene-queryparser-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/lucene-join-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/lucene-sandbox-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/antlr-runtime-3.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-commons-4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/concurrentlinkedhashmap-lru-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hppc-0.5.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/spatial4j-0.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150729_173002_666.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150729_173002_666.suites
   [junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 240. Very likely a JVM crash.  Process output piped in logs above.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1484)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:734: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:671: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:517: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1447: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1002: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 295 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 6 artifacts
Archive block size is 32768
Received 0 blocks and 1851618101 bytes
Compression is 0.0%
Took 5 min 21 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1045 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1045/

10 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1721, name=Thread-1644, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1721, name=Thread-1644, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1719, name=Thread-1642, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1719, name=Thread-1642, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1722, name=Thread-1645, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1722, name=Thread-1645, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1723, name=Thread-1646, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1723, name=Thread-1646, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:111)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1717, name=Thread-1640, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1717, name=Thread-1640, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1720, name=Thread-1643, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1720, name=Thread-1643, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1718, name=Thread-1641, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1718, name=Thread-1641, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1714, name=Thread-1637, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1714, name=Thread-1637, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)




Build Log:
[...truncated 1678 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterThreadsToSegments
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1642,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1644,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1640,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1643,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1641,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1637,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1645,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1646,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:111)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 4:08:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.index.TestIndexWriterThreadsToSegments
   [junit4]   2>    1) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:601)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:450)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2>    2) Thread[id=1716, name=Thread-1639, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2>    3) Thread[id=1698, name=SUITE-TestIndexWriterThreadsToSegments-seed#[F72A04E52535E596], state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:591)
   [junit4]   2>    4) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2>    5) Thread[id=1715, name=Thread-1638, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2>    6) Thread[id=1699, name=TEST-TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom-seed#[F72A04E52535E596], state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom(TestIndexWriterThreadsToSegments.java:226)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterThreadsToSegments -Dtests.method=testSegmentCountOnFlushRandom -Dtests.seed=F72A04E52535E596 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sr_RS_#Latn -Dtests.timezone=America/Santa_Isabel -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   7199s J1 | TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> dec 12, 2015 4:08:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> dec 12, 2015 4:09:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 3 threads leaked from SUITE scope at org.apache.lucene.index.TestIndexWriterThreadsToSegments: 
   [junit4]   2>    1) Thread[id=1715, name=Thread-1638, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2>    2) Thread[id=1699, name=TEST-TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom-seed#[F72A04E52535E596], state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom(TestIndexWriterThreadsToSegments.java:226)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=1716, name=Thread-1639, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2> dec 12, 2015 4:09:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=1715, name=Thread-1638, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>    2) Thread[id=1699, name=TEST-TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom-seed#[F72A04E52535E596], state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>    3) Thread[id=1716, name=Thread-1639, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001
   [junit4]   2> Dec 13, 2015 12:09:02 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {field=Lucene50(blocksize=128)}, docValues:{}, sim=DefaultSimilarity, locale=sr_RS_#Latn, timezone=America/Santa_Isabel
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=130871024,total=402128896
   [junit4]   2> NOTE: All tests run in this JVM: [TestTermScorer, TestDocCount, TestSortedSetSortField, TestIndexWriterLockRelease, TestIndexWriterExceptions, TestBoostQuery, TestDisjunctionMaxQuery, TestIndexWriterDeleteByQuery, Test2BNumericDocValues, TestSegmentMerger, TestDemoParallelLeafReader, TestFlex, TestFastDecompressionMode, TestBoolean2, TestControlledRealTimeReopenThread, TestTermVectorsReader, TestRecyclingIntBlockAllocator, TestBlendedTermQuery, TestFSTs, TestFieldValueFilter, TestIndexWriterThreadsToSegments]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterThreadsToSegments -Dtests.seed=F72A04E52535E596 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sr_RS_#Latn -Dtests.timezone=America/Santa_Isabel -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestIndexWriterThreadsToSegments (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1721, name=Thread-1644, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1719, name=Thread-1642, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	... 2 moreThrowable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1722, name=Thread-1645, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1723, name=Thread-1646, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:111)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thr

[...truncated too long message...]

> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	... 2 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1718, name=Thread-1641, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	... 2 moreThrowable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1714, name=Thread-1637, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4] Completed [366/414 (1!)] on J1 in 7220.20s, 3 tests, 10 errors <<< FAILURES!

[...truncated 170 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 414 suites (1 ignored), 3345 tests, 9 suite-level errors, 1 error, 29 ignored (25 assumptions) [seed: F72A04E52535E596]

Total time: 157 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1044 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1044/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:41741/_v/vj: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41741/_v/vj: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:295)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:412)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#4): CloudJettyRunner [url=http://127.0.0.1:56785/be/c/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#4): CloudJettyRunner [url=http://127.0.0.1:56785/be/c/collection1]
	at __randomizedtesting.SeedInfo.seed([91E270C9DE67D054:19B64F13709BBDAC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10372 lines...]
   [junit4] Suite: org.apache.solr.cloud.RollingRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/init-core-data-001
   [junit4]   2> 169612 INFO  (SUITE-RollingRestartTest-seed#[91E270C9DE67D054]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 169613 INFO  (SUITE-RollingRestartTest-seed#[91E270C9DE67D054]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /be/c
   [junit4]   2> 169627 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 169628 INFO  (Thread-3945) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 169628 INFO  (Thread-3945) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 169728 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkTestServer start zk server on port:55353
   [junit4]   2> 169729 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 169758 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 169762 INFO  (zkCallback-83-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@16b7755f name:ZooKeeperConnection Watcher:127.0.0.1:55353 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 169762 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 169762 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 169762 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 169766 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 169767 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 169773 INFO  (zkCallback-84-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2917ca48 name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 169773 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 169774 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 169774 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 169777 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 169779 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 169781 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 169783 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 169783 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 169788 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 169788 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 169791 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 169791 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 169794 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 169794 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 169799 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 169799 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 169802 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 169803 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 169805 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 169805 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 169807 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 169807 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 169810 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 169810 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 169812 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 169812 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 169814 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 169814 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 170178 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1
   [junit4]   2> 170180 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 170183 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ce1ccdd{/be/c,null,AVAILABLE}
   [junit4]   2> 170184 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6c50f0ce{HTTP/1.1}{127.0.0.1:39963}
   [junit4]   2> 170184 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server Started @173124ms
   [junit4]   2> 170184 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/be/c, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/tempDir-001/control/data, hostPort=39963}
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4dfd245f
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001'
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 170186 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 170186 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 170188 INFO  (zkCallback-85-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66490ef3 name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 170188 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 170188 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 170190 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 170190 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/solr.xml
   [junit4]   2> 170204 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores
   [junit4]   2> 170205 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer New CoreContainer 1784043863
   [junit4]   2> 170205 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001]
   [junit4]   2> 170205 WARN  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/lib
   [junit4]   2> 170205 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 170206 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55353/solr
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 170208 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 170210 INFO  (zkCallback-87-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1323271c name:ZooKeeperConnection Watcher:127.0.0.1:55353 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 170210 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 170211 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 170214 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 170216 INFO  (zkCallback-88-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e13f60e name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 170216 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 170218 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 170220 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 170224 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 170227 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 170229 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 170232 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 170235 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 170237 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 170239 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 170241 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39963_be%2Fc
   [junit4]   2> 170242 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39963_be%2Fc
   [junit4]   2> 170244 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 170246 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 170248 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 170249 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000
   [junit4]   2> 170250 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39963_be%2Fc
   [junit4]   2> 170250 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 170252 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer Overseer (id=95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) starting
   [junit4]   2> 170254 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 170275 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 170275 INFO  (OverseerCollectionConfigSetProcessor-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 170276 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 170278 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 170314 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 170315 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 170316 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores
   [junit4]   2> 170317 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, transient=false, configSetProperties=configsetprops.json, schema=schema.xml, dataDir=data/, collection=control_collection, name=collection1, shard=, config=solrconfig.xml}
   [junit4]   2> 170317 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1
   [junit4]   2> 170318 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 170319 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 170319 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 170320 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 170320 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 170324 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"} current state version: 0
   [junit4]   2> 170324 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"}
   [junit4]   2> 170325 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 170325 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 170325 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 170328 INFO  (zkCallback-88-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 171325 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 171326 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 171327 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 171327 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1'
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 171331 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 171339 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 171354 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 171384 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 171394 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 171670 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 171679 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 171682 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 171705 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 171710 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 171713 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 171714 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 171714 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 171714 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 171715 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 171715 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 171716 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 171717 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 171717 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 171724 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/]
   [junit4]   2> 171724 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@531fb850
   [junit4]   2> 171725 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data
   [junit4]   2> 171726 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index/
   [junit4]   2> 171726 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 171726 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index
   [junit4]   2> 171727 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=93.4091796875, floorSegmentMB=1.140625, forceMergeDeletesPctAllowed=19.040737700900713, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.798895882218802
   [junit4]   2> 171735 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 171735 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 171736 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/
   [junit4]   2> 171742 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 171742 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 171744 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 171744 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 171748 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171749 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171750 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171752 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171757 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/admin/luke,/admin/mbeans,/dump,/admin/segments,/update/csv,standard,/schema,/replication,/config,/admin/properties,/update,/admin/system,/get,/update/json/docs,/admin/logging,/admin/ping,/update/json,/admin/threads,/admin/file
   [junit4]   2> 171759 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 171761 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=73.013671875, floorSegmentMB=1.62890625, forceMergeDeletesPctAllowed=22.559599334711724, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1a72b5ed[collection1] main
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 171764 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 171764 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 171765 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1520326656622854144
   [junit4]   2> 171769 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1520326656622854144
   [junit4]   2> 171769 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 171770 INFO  (searcherExecutor-285-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1a72b5ed[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 171770 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 171771 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:39963/be/c collection:control_collection shard:shard1
   [junit4]   2> 171773 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 171778 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95020415899533316-core_node1-n_0000000000
   [junit4]   2> 171779 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 171781 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39963/be/c/collection1/
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39963/be/c/collection1/ has no replicas
   [junit4]   2> 171782 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 171783 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 171788 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 171791 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39963/be/c/collection1/ shard1
   [junit4]   2> 171792 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 171821 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 171822 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 171824 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35fda302 name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 171824 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 171824 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 171824 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 171829 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 171896 INFO  (zkCallback-88-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 171896 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:39963/be/c/collection1/ and leader is http://127.0.0.1:39963/be/c/collection1/
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39963/be/c
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 171946 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"} current state version: 2
   [junit4]   2> 171947 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"}
   [junit4]   2> 172051 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 172051 INFO  (zkCallback-88-thread-2-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 172190 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores/collection1
   [junit4]   2> 172191 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001
   [junit4]   2> 172191 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 172194 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3951e1e6{/be/c,null,AVAILABLE}
   [junit4]   2> 172194 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4111f10d{HTTP/1.1}{127.0.0.1:57469}
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server Started @175134ms
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=57469, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores, hostContext=/be/c, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/tempDir-001/jetty1}
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4dfd245f
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001'
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 172196 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 172196 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 172198 INFO  (zkCallback-91-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32c8d2ed name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 172198 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 172199 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 172200 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 172200 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/solr.xml
   [junit4]   2> 172217 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores
   [junit4]   2> 172218 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer New CoreContainer 1400400768
   [junit4]   2> 172218 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001]
   [junit4]   2> 172218 WARN  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/lib
   [junit4]   2> 172219 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 172220 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55353/solr
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 172222 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 172223 INFO  (zkCallback-93-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73c68f60 name:ZooKeeperConnection Watcher:127.0.0.1:55353 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 172224 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 172224 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 172227 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 172230 INFO  (zkCallback-94-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7212f0ad name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 172230 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 172238 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 173247 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57469_be%2Fc
   [junit4]   2> 173247 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57469_be%2Fc
   [junit4]   2> 173249 INFO  (zkCallback-94-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 173250 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 173250 INFO  (zkCallback-88-thread-2-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 173251 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 173252 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95020415899533320-127.0.0.1:57469_be%2Fc-n_0000000001
   [junit4]   2> 173254 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000 to know if I could be the leader
   [junit4]   2> 173275 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 173275 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 173276 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores
   [junit4]   2> 173277 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, config=solrconfig.xml, transient=false, name=collection1, shard=, collection=collection1, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json}
   [junit4]   2> 173277 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores/collection1
   [junit4]   2> 173278 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 173280 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 173280 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 173280 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 173280 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 173283 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:57469_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:57469/be/c"} current state version: 3
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:57469_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:57469/be/c"}
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 173389 INFO  (zkCallback-88-thread-2-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 173389 INFO  (zkCallback-94-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 173389 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 174284 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 174285 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 174285 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 174285 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 174286 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 174286 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores/collection1'
   [junit4]   2> 174287 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 174287 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 174290 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 174298 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 174315 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 174362 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 174371 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 174633 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 174643 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 174646 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 174663 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 174668 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 174671 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 174672 WARN  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 174672 WARN  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 174673 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:5

[...truncated too long message...]

p://127.0.0.1:48329/_v/vj, http://127.0.0.1:44086/_v/vj, http://127.0.0.1:35231/_v/vj, http://127.0.0.1:46864/_v/vj]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([91E270C9DE67D054]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:41741/_v/vj, http://127.0.0.1:48329/_v/vj, http://127.0.0.1:44086/_v/vj, http://127.0.0.1:35231/_v/vj, http://127.0.0.1:46864/_v/vj]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46864/_v/vj
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 more
   [junit4]   2> 1977347 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[91E270C9DE67D054]-worker) [n:127.0.0.1:44086__v%2Fvj c:awholynewstresscollection_collection5_1 s:shard5 r:core_node12 x:awholynewstresscollection_collection5_1_shard5_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91E270C9DE67D054-001
   [junit4]   2> Dec 12, 2015 5:31:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=4892, maxDocsPerChunk=810, blockSize=4), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=4892, blockSize=4)), sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ko_KR, timezone=Africa/Porto-Novo
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=92832704,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [MBeansHandlerTest, ZkCLITest, ChaosMonkeySafeLeaderTest, OutOfBoxZkACLAndCredentialsProvidersTest, UpdateRequestProcessorFactoryTest, QueryResultKeyTest, TestQueryUtils, SpellPossibilityIteratorTest, TestCollapseQParserPlugin, TestFastLRUCache, TestDFRSimilarityFactory, DocumentBuilderTest, TestSolrCloudWithKerberosAlt, DeleteLastCustomShardedReplicaTest, ChaosMonkeyNothingIsSafeTest, TestManagedResource, SortSpecParsingTest, DirectSolrSpellCheckerTest, TestLMJelinekMercerSimilarityFactory, DocExpirationUpdateProcessorFactoryTest, ClusterStateUpdateTest, TestTrie, TestFileDictionaryLookup, SpellCheckCollatorTest, TestQueryTypes, FastVectorHighlighterTest, HdfsDirectoryTest, TestFieldTypeResource, ConnectionManagerTest, SolrInfoMBeanTest, SchemaVersionSpecificBehaviorTest, HdfsThreadLeakTest, TestQuerySenderListener, DistribCursorPagingTest, TermsComponentTest, WordBreakSolrSpellCheckerTest, TimeZoneUtilsTest, ConvertedLegacyTest, StressHdfsTest, TestReplicationHandlerBackup, TestLFUCache, DistributedFacetPivotSmallAdvancedTest, TestDistribDocBasedVersion, NotRequiredUniqueKeyTest, TestMiniSolrCloudClusterBase, TestScoreJoinQPNoScore, TestTrieFacet, SolrIndexConfigTest, ExplicitHLLTest, SuggesterFSTTest, TestSchemaResource, RecoveryAfterSoftCommitTest, SolrPluginUtilsTest, TestReRankQParserPlugin, TestExactStatsCache, TestMaxScoreQueryParser, TestOrdValues, SolrXmlInZkTest, SharedFSAutoReplicaFailoverUtilsTest, BlockCacheTest, TestOverriddenPrefixQueryForCustomFieldType, TestSchemaManager, FileUtilsTest, TestCloudSchemaless, TestFieldSortValues, MultiTermTest, AssignTest, OpenExchangeRatesOrgProviderTest, ZkSolrClientTest, TestCodecSupport, TestFieldCollectionResource, BasicFunctionalityTest, TestChildDocTransformer, TestSimpleTrackingShardHandler, TestCursorMarkWithoutUniqueKey, DistributedTermsComponentTest, CloudMLTQParserTest, SoftAutoCommitTest, ShardRoutingTest, AddSchemaFieldsUpdateProcessorFactoryTest, DistanceFunctionTest, DocValuesMissingTest, TestBulkSchemaConcurrent, TestCloudPivotFacet, DistributedMLTComponentTest, ConjunctionSolrSpellCheckerTest, SimplePostToolTest, TestRandomForceMergePolicy, TermVectorComponentTest, TestBinaryResponseWriter, BasicDistributedZkTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [239/555 (2!)] on J1 in 532.01s, 1 test, 1 error <<< FAILURES!

[...truncated 1008 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151212_045830_148.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid7675.hprof ...
   [junit4] Heap dump file created [613583618 bytes in 8.932 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 23 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 555 suites (7 ignored), 2192 tests, 1 error, 1 failure, 33 ignored (18 assumptions) [seed: 91E270C9DE67D054]

Total time: 225 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1043 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1043/

2 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}]> but was:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}, {indexVersion=1449825644507,generation=3,filelist=[_2nvt.fdt, _2nvt.fdx, _2nvt.fnm, _2nvt.nvd, _2nvt.nvm, _2nvt.si, _2nvt_MockRandom_0.doc, _2nvt_MockRandom_0.sd, _2nvt_MockRandom_0.tmp, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxg.fdt, _2nxg.fdx, _2nxg.fnm, _2nxg.nvd, _2nxg.nvm, _2nxg.si, _2nxg_MockRandom_0.doc, _2nxg_MockRandom_0.sd, _2nxg_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}]> but was:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}, {indexVersion=1449825644507,generation=3,filelist=[_2nvt.fdt, _2nvt.fdx, _2nvt.fnm, _2nvt.nvd, _2nvt.nvm, _2nvt.si, _2nvt_MockRandom_0.doc, _2nvt_MockRandom_0.sd, _2nvt_MockRandom_0.tmp, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxg.fdt, _2nxg.fdx, _2nxg.fnm, _2nxg.nvd, _2nxg.nvm, _2nxg.si, _2nxg_MockRandom_0.doc, _2nxg_MockRandom_0.sd, _2nxg_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([252FCDCEA534147F:F8D6FED57C1A7C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1140)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Fri Dec 11 22:22:02 GMT+13:00 2015

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Fri Dec 11 22:22:02 GMT+13:00 2015
	at __randomizedtesting.SeedInfo.seed([252FCDCEA534147F:FE84CD08A01C7DCC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1419)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:771)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10887 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/init-core-data-001
   [junit4]   2> 1107487 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestIndexAndConfigReplication
   [junit4]   2> 1107487 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1
   [junit4]   2> 1107491 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1107493 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16de7d5f{/solr,null,AVAILABLE}
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68f8f3b3{HTTP/1.1}{127.0.0.1:35053}
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server Started @1110494ms
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=35053, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/data, hostContext=/solr}
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001'
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1107495 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1107495 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/solr.xml
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/.
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1347987264
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001]
   [junit4]   2> 1107510 WARN  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/lib
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1107511 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1107513 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/.
   [junit4]   2> 1107514 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, loadOnStartup=true, config=solrconfig.xml, configSetProperties=configsetprops.json, name=collection1, dataDir=data/, schema=schema.xml}
   [junit4]   2> 1107514 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1
   [junit4]   2> 1107514 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1107515 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1'
   [junit4]   2> 1107515 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1107515 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1107515 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1107515 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1107523 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1107536 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1107561 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1107565 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1107573 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1'
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 1107576 WARN  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1107576 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 1107576 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1187930328, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1107577 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@49765f6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6fa6fbd4),segFN=segments_1,generation=1}
   [junit4]   2> 1107577 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1107577 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1107578 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1107581 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107583 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107584 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107585 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107601 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1107601 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/schema,/update/json/docs,/admin/system,standard,lazy,/admin/logging,/admin/properties,/update/json,/admin/plugins,/get,/replication,/admin/file,/admin/mbeans,defaults,/admin/segments,/update/csv,/update,/admin/luke,/config,/admin/ping
   [junit4]   2> 1107602 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@40455297
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@49765f6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6fa6fbd4),segFN=segments_1,generation=1}
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@37bf255[collection1] main
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/conf
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/conf
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1107604 INFO  (searcherExecutor-2298-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@37bf255[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1108016 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1
   [junit4]   2> 1108020 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62dcc762{/solr,null,AVAILABLE}
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6ff0f6d1{HTTP/1.1}{127.0.0.1:46287}
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server Started @1111022ms
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=46287}
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002'
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/solr.xml
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/.
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer New CoreContainer 974364157
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002]
   [junit4]   2> 1108037 WARN  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/lib
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1108040 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/.
   [junit4]   2> 1108041 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, dataDir=data/, config=solrconfig.xml, loadOnStartup=true, configSetProperties=configsetprops.json, name=collection1, schema=schema.xml}
   [junit4]   2> 1108041 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1
   [junit4]   2> 1108041 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1108042 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1'
   [junit4]   2> 1108042 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1108042 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1108042 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1108042 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1108050 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1108063 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1108089 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1108094 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1108101 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1108103 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1'
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1108105 WARN  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1108105 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1108105 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1187930328, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1108106 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a08ff5b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@34b4c4ec),segFN=segments_1,generation=1}
   [junit4]   2> 1108106 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1108106 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1108108 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1108111 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108112 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108113 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108114 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108116 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1108116 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/schema,/update/json/docs,/admin/system,standard,lazy,/admin/logging,/admin/properties,/update/json,/admin/plugins,/get,/replication,/admin/file,/admin/mbeans,defaults,/admin/segments,/update/csv,/update,/admin/luke,/config,/admin/ping
   [junit4]   2> 1108117 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7155e71f
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a08ff5b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@34b4c4ec),segFN=segments_1,generation=1}
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@41d39a0e[collection1] main
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/conf
   [junit4]   2> 1108119 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1108119 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/conf
   [junit4]   2> 1108119 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1108121 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1108121 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1108121 INFO  (searcherExecutor-2308-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@41d39a0e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1108124 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wt=javabin&qt=/replication&command=indexversion&version=2} status=0 QTime=0 
   [junit4]   2> 1108545 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={wt=javabin&version=2&sort=id+desc&q=*:*} hits=0 status=0 QTime=0 
   [junit4]   2> 1108549 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 1
   [junit4]   2> 1108553 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 1
   [junit4]   2> 1108556 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 1
   [junit4]   2> 1108559 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 1
   [junit4]   2> 1108562 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 0
   [junit4]   2> 1108566 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 1
   [junit4]   2> 1108568 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 0
   [junit4]   2> 1108597 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 1
   [junit4]   2> 1108599 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 0
   [junit4]   2> 1108602 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 1
   [junit4]   2> 1108605 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 0
   [junit4]   2> 1108607 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[11]} 0 1
   [junit4]   2> 1108610 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 0
   [junit4]   2> 1108613 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 1
   [junit4]   2> 1108615 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 0
   [junit4]   2> 1108618 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 1
   [junit4]   2> 1108620 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 0
   [junit4]   2> 1108623 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 1
   [junit4]   2> 1108626 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 0
   [junit4]   2> 1108629 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 1
   [junit4]   2> 1108631 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 0
   [junit4]   2> 1108634 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 1
   [junit4]   2> 1108637 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 0
   [junit4]   2> 1108640 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 1
   [junit4]   2> 1108643 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 0
   [junit4]   2> 1108645 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 1
   [junit4]   2> 1108647 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26]} 0 0
   [junit4]   2> 1108650 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27]} 0 1
   [junit4]   2> 1108652 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[28]} 0 0
   [junit4]   2> 1108655 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[29]} 0 1
   [junit4]   2> 1108657 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[30]} 0 0
   [junit4]   2> 1108660 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[31]} 0 1
   [junit4]   2> 1108662 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[32]} 0 0
   [junit4]   2> 1108665 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33]} 0 1
   [junit4]   2> 1108667 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[34]} 0 0
   [junit4]   2> 1108670 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35]} 0 1
   [junit4]   2> 1108672 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[36]} 0 0
   [junit4]   2> 1108675 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[37]} 0 1
   [junit4]   2> 1108677 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[38]} 0 0
   [junit4]   2> 1108680 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[39]} 0 1
   [junit4]   2> 1108682 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40]} 0 0
   [junit4]   2> 1108685 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[41]} 0 1
   [junit4]   2> 1108687 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[42]} 0 0
   [junit4]   2> 1108695 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[43]} 0 5
   [junit4]   2> 1108697 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[44]} 0 0
   [junit4]   2> 1108700 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[45]} 0 1
   [junit4]   2> 1108702 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[46]} 0 0
   [junit4]   2> 1108705 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47]} 0 1
   [junit4]   2> 1108708 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[48]} 0 0
   [junit4]   2> 1108711 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49]} 0 1
   [junit4]   2> 1108713 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50]} 0 0
   [junit4]   2> 1108717 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[51]} 0 1
   [junit4]   2> 1108719 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[52]} 0 0
   [junit4]   2> 1108722 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[53]} 0 1
   [junit4]   2> 1108724 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[54]} 0 0
   [junit4]   2> 1108727 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[55]} 0 1
   [junit4]   2> 1108729 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[56]} 0 0
   [junit4]   2> 1108732 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[57]} 0 1
   [junit4]   2> 1108734 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[58]} 0 0
   [junit4]   2> 1108738 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[59]} 0 1
   [junit4]   2> 1108740 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[60]} 0 0
   [junit4]   2> 1108803 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[61]} 0 61
   [junit4]   2> 1108806 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[62]} 0 0
   [junit4]   2> 1108812 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[63]} 0 4
   [junit4]   2> 1108814 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64]} 0 0
   [junit4]   2> 1108818 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[65]} 0 2
   [junit4]   2> 1108820 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66]} 0 0
   [junit4]   2> 1108824 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[67]} 0 1
   [junit4]   2> 1108826 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68]} 0 0
   [junit4]   2> 1108829 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[69]} 0 1
   [junit4]   2> 1108831 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[70]} 0 0
   [junit4]   2> 1108837 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[71]} 0 4
   [junit4]   2> 1108840 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[72]} 0 0
   [junit4]   2> 1108842 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[73]} 0 1
   [junit4]   2> 1108845 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[74]} 0 0
   [junit4]   2> 1108851 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[75]} 0 3
   [junit4]   2> 1108853 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[76]} 0 0
   [junit4]   2> 1108856 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[77]} 0 1
   [junit4]   2> 1108858 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[78]} 0 0
   [junit4]   2> 1108861 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[79]} 0 1
   [junit4]   2> 1108863 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[80]} 0 0
   [junit4]   2> 1108866 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[81]} 0 1
   [junit4]   2> 1108868 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[82]} 0 0
   [junit4]   2> 1108871 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[83]} 0 1
   [junit4]   2> 1108873 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[84]} 0 0
   [junit4]   2> 1108875 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[85]} 0 1
   [junit4]   2> 1108877 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[86]} 0 0
   [junit4]   2> 1108880 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[87]} 0 1
   [junit4]   2> 1108882 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[88]} 0 0
   [junit4]   2> 1108885 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[89]} 0 1
   [junit4]   2> 1108887 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[90]} 0 0
   [junit4]   2> 1108890 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[91]} 0 1
   [junit4]   2> 1108893 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[92]} 0 0
   [junit4]   2> 1108895 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[93]} 0 1
   [junit4]   2> 1108897 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[94]} 0 0
   [junit4]   2> 1108900 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[95]} 0 1
   [junit4]   2> 1108902 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[96]} 0 0
   [junit4]   2> 1108905 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[97]} 0 1
   [junit4]   2> 1108907 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[98]} 0 0
   [junit4]   2> 1108910 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[99]} 0 1
   [junit4]   2> 1108912 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[100]} 0 0
   [junit4]   2> 1108916 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[101]} 0 1
   [junit4]   2> 1108919 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[102]} 0 1
   [junit4]   2> 1108929 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[103]} 0 8
   [junit4]   2> 1108931 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[104]} 0 0
   [junit4]   2> 1108936 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[105]} 0 3
   [junit4]   2> 1108939 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[106]} 0 0
   [junit4]   2> 1108943 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[107]} 0 2
   [junit4]   2> 1108994 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[108]} 0 49
   [junit4]   2> 1108997 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[109]} 0 1
   [junit4]   2> 1108999 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[110]} 0 0
   [junit4]   2> 1109002 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[111]} 0 1
   [junit4]   2> 1109004 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[112]} 0 0
   [junit4]   2> 1109007 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[113]} 0 1
   [junit4]   2> 1109009 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[114]} 0 0
   [junit4]   2> 1109012 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[115]} 0 0
   [junit4]   2> 1109014 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[116]} 0 0
   [junit4]   2> 1109017 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[117]} 0 0
   [junit4]   2> 1109019 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[118]} 0 0
   [junit4]   2> 1109021 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[119]} 0 1
   [junit4]   2> 1109024 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [c

[...truncated too long message...]

dler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data
   [junit4]   2> 2123296 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data/index;done=false>>]
   [junit4]   2> 2123296 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data/index
   [junit4]   2> 2123296 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5f966a00{/solr,null,UNAVAILABLE}
   [junit4]   2> 2123297 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@20a391fd{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2123297 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1069535444
   [junit4]   2> 2123298 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@39384d4e
   [junit4]   2> 2123298 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2123298 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data/index.20151211222255571 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data/index.20151211222255571;done=false>>]
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data/index.20151211222255571
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data;done=false>>]
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data
   [junit4]   2> 2123300 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@b548fe8{/solr,null,UNAVAILABLE}
   [junit4]   2> 2123301 INFO  (SUITE-TestReplicationHandler-seed#[252FCDCEA534147F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {newname=PostingsFormat(name=MockRandom), name=PostingsFormat(name=MockRandom), id=PostingsFormat(name=MockRandom)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=fr, timezone=Etc/GMT-13
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=149649120,total=530055168
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolr4Spatial, BasicAuthIntegrationTest, TestValueSourceCache, TestRestManager, TestRandomDVFaceting, TestDocBasedVersionConstraints, UUIDFieldTest, TestSortingResponseWriter, DistributedFacetPivotWhiteBoxTest, DefaultValueUpdateProcessorTest, TestShardHandlerFactory, RankQueryTest, HLLUtilTest, TestRTGBase, TestPKIAuthenticationPlugin, TestSolrQueryParserResource, TestBlendedInfixSuggestions, IndexSchemaTest, VersionInfoTest, TestManagedSchema, RAMDirectoryFactoryTest, BigEndianAscendingWordDeserializerTest, TestCSVLoader, TestNoOpRegenerator, ReturnFieldsTest, TestShortCircuitedRequests, TestMiniSolrCloudCluster, TestFaceting, TestStressLucene, TestHighFrequencyDictionaryFactory, TestRealTimeGet, ResponseHeaderTest, UUIDUpdateProcessorFallbackTest, CopyFieldTest, TestBinaryField, TestFoldingMultitermQuery, TestDistributedSearch, TestReplicaProperties, DistributedSuggestComponentTest, ResponseLogComponentTest, CloudExitableDirectoryReaderTest, StandardRequestHandlerTest, OverseerTaskQueueTest, DeleteShardTest, JsonLoaderTest, DocumentAnalysisRequestHandlerTest, SynonymTokenizerTest, TestLRUCache, OutputWriterTest, TestManagedSynonymFilterFactory, DistributedSpellCheckComponentTest, OverseerCollectionConfigSetProcessorTest, TestJsonRequest, DateFieldTest, TestDocSet, CacheHeaderTest, TestAtomicUpdateErrorCases, PreAnalyzedFieldTest, EchoParamsTest, DisMaxRequestHandlerTest, HdfsChaosMonkeyNothingIsSafeTest, HdfsWriteToMultipleCollectionsTest, AddBlockUpdateTest, ParsingFieldUpdateProcessorsTest, AnalyticsQueryTest, CollectionTooManyReplicasTest, TestStressUserVersions, HdfsDirectoryFactoryTest, DeleteInactiveReplicaTest, InfoHandlerTest, TestReplicationHandler]
   [junit4] Completed [217/554 (1!)] on J0 in 1015.84s, 13 tests, 2 failures <<< FAILURES!

[...truncated 1071 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 554 suites (7 ignored), 2188 tests, 2 failures, 33 ignored (18 assumptions) [seed: 252FCDCEA534147F]

Total time: 215 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1042 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1042/

1 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
	at __randomizedtesting.SeedInfo.seed([5DB3A1C37B327030:DAE4DC4CEA6B0CB0]:0)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:427)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3161)
	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3147)
	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3124)
	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3087)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1741)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:596)
	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)




Build Log:
[...truncated 8481 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=5DB3A1C37B327030 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=Etc/Universal -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    126s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([5DB3A1C37B327030:DAE4DC4CEA6B0CB0]:0)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:427)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]    > 	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
   [junit4]    > 	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
   [junit4]    > 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3161)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3147)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3124)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3087)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1741)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:596)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/J1/temp/lucene.search.TestGeoPointQuery_5DB3A1C37B327030-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {point=PostingsFormat(name=Direct), id=PostingsFormat(name=Memory doPackFST= false), string=Lucene50(blocksize=128)}, docValues:{point=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=el, timezone=Etc/Universal
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=196115464,total=403701760
   [junit4]   2> NOTE: All tests run in this JVM: [DuplicateFilterTest, TestGeoPointQuery]
   [junit4] Completed [18/20 (1!)] on J1 in 386.20s, 23 tests, 1 error <<< FAILURES!

[...truncated 4 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/temp/junit4-J1-20151210_164328_858.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18413.hprof ...
   [junit4] Heap dump file created [552835499 bytes in 5.683 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 20 suites, 142 tests, 1 error [seed: 5DB3A1C37B327030]

Total time: 116 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1041 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1041/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([BF6DB73668EA9BBA:373988ECC616F642]:0)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.getCoreContainer(JettySolrRunner.java:374)
	at org.apache.solr.cloud.ChaosMonkey.expireSession(ChaosMonkey.java:124)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10273 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/init-core-data-001
   [junit4]   2> 703861 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 703950 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 703966 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 703977 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 703992 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_41510_hdfs____.y4dasf/webapp
   [junit4]   2> 704148 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 704475 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41510
   [junit4]   2> 704620 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 704622 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 704640 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40855_datanode____4gus3n/webapp
   [junit4]   2> 704832 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 705128 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40855
   [junit4]   2> 705246 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 705251 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 705265 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59784_datanode____.n9srii/webapp
   [junit4]   2> 705465 INFO  (IPC Server handler 3 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-13b02519-4d94-4f73-a6d3-618de83dacac node DatanodeRegistration(127.0.0.1, datanodeUuid=ca1780d1-b226-41a3-926a-10724401377c, infoPort=40855, ipcPort=36581, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 705465 INFO  (IPC Server handler 3 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-434609a3-3092-4032-8859-72cd30a2283f node DatanodeRegistration(127.0.0.1, datanodeUuid=ca1780d1-b226-41a3-926a-10724401377c, infoPort=40855, ipcPort=36581, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 705492 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 705688 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59784
   [junit4]   2> 705900 INFO  (IPC Server handler 5 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-aa26c54e-7058-44a3-bfd9-8d922be4a1d2 node DatanodeRegistration(127.0.0.1, datanodeUuid=6300e33a-b4ca-4d3b-8167-df5a8e970f8f, infoPort=59784, ipcPort=37865, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 705900 INFO  (IPC Server handler 5 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-b9f7b83a-4810-4bba-bd5c-bfba988f8972 node DatanodeRegistration(127.0.0.1, datanodeUuid=6300e33a-b4ca-4d3b-8167-df5a8e970f8f, infoPort=59784, ipcPort=37865, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 706082 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 706084 INFO  (Thread-3618) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 706084 INFO  (Thread-3618) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 706184 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkTestServer start zk server on port:34302
   [junit4]   2> 706184 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706184 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706192 INFO  (zkCallback-1573-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6911a79 name:ZooKeeperConnection Watcher:127.0.0.1:34302 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706192 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706192 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706192 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 706200 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706210 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706210 INFO  (zkCallback-1574-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56846c0b name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706210 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706211 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706211 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 706214 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 706216 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 706217 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 706219 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 706219 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 706221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 706221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 706223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 706223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 706224 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 706224 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 706226 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 706226 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 706227 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 706227 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 706229 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 706229 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 706230 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 706230 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 706232 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 706232 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 706234 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 706234 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 706235 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 706236 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 706608 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1
   [junit4]   2> 706610 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 706618 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a93d624{/,null,AVAILABLE}
   [junit4]   2> 706618 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d7d44cb{HTTP/1.1}{127.0.0.1:46663}
   [junit4]   2> 706618 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server Started @709456ms
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores, solr.data.dir=hdfs://localhost:58696/hdfs__localhost_58696__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001_tempDir-002_control_data, hostPort=46663, hostContext=/}
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001'
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706620 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706628 INFO  (zkCallback-1575-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7dd6c24 name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706629 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706629 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706632 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15185de05b70002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 706632 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 706632 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/solr.xml
   [junit4]   2> 706653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores
   [junit4]   2> 706653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer New CoreContainer 134746427
   [junit4]   2> 706653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001]
   [junit4]   2> 706653 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/lib
   [junit4]   2> 706654 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 706655 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34302/solr
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706658 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706658 INFO  (zkCallback-1577-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37708dc5 name:ZooKeeperConnection Watcher:127.0.0.1:34302 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706660 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706660 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706664 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706680 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50d1b0da name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706680 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706682 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 706684 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 706687 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 706689 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 706691 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 706696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 706701 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 706704 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 706706 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 706711 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46663_
   [junit4]   2> 706711 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46663_
   [junit4]   2> 706714 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 706718 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 706720 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 706720 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95004255776604164-127.0.0.1:46663_-n_0000000000
   [junit4]   2> 706721 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46663_
   [junit4]   2> 706721 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 706722 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer Overseer (id=95004255776604164-127.0.0.1:46663_-n_0000000000) starting
   [junit4]   2> 706727 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 706740 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 706742 INFO  (OverseerCollectionConfigSetProcessor-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 706744 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 706744 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 706779 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 706779 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 706781 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores
   [junit4]   2> 706782 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {shard=, configSetProperties=configsetprops.json, collection=control_collection, loadOnStartup=true, dataDir=data/, config=solrconfig.xml, transient=false, schema=schema.xml, coreNodeName=, name=collection1}
   [junit4]   2> 706782 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1
   [junit4]   2> 706782 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 706789 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 706789 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 706789 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 706789 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 706793 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 706794 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"} current state version: 0
   [junit4]   2> 706794 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"}
   [junit4]   2> 706795 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 706795 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 706798 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 707794 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 707795 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 707796 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 707796 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1'
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 707807 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 707817 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 707830 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 707863 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 707878 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 708284 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 708296 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 708298 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 708322 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 708327 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 708330 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 708331 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 708331 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 708331 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 708332 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 708332 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 708333 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 708333 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 708333 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 708344 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58696/solr_hdfs_home
   [junit4]   2> 708344 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 708344 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 708345 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64c3d9c0
   [junit4]   2> 708345 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 708434 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 708449 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 708477 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 708477 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 708505 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 708506 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 708506 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 708559 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 708559 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 708559 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=1.5556640625, floorSegmentMB=2.083984375, forceMergeDeletesPctAllowed=2.632039295550377, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 708615 INFO  (IPC Server handler 5 on 58696) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36944 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-13b02519-4d94-4f73-a6d3-618de83dacac:NORMAL:127.0.0.1:42534|RBW], ReplicaUnderConstruction[[DISK]DS-aa26c54e-7058-44a3-bfd9-8d922be4a1d2:NORMAL:127.0.0.1:36944|RBW]]} size 0
   [junit4]   2> 708619 INFO  (IPC Server handler 6 on 58696) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42534 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-aa26c54e-7058-44a3-bfd9-8d922be4a1d2:NORMAL:127.0.0.1:36944|RBW], ReplicaUnderConstruction[[DISK]DS-434609a3-3092-4032-8859-72cd30a2283f:NORMAL:127.0.0.1:42534|FINALIZED]]} size 0
   [junit4]   2> 708632 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71de19d8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 708632 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 708635 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 708639 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 708640 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 708640 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 708640 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 708641 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 708644 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708645 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708655 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708663 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708670 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/system,/admin/properties,/get,/update/csv,/admin/logging,/admin/file,/update/json,/admin/plugins,standard,/update/json/docs,/admin/segments,/admin/ping,/update,/schema,/admin/mbeans,/replication,/config,/admin/luke,/dump
   [junit4]   2> 708672 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 708678 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 708679 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 708702 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 708702 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 708704 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=32.3037109375, floorSegmentMB=0.3125, forceMergeDeletesPctAllowed=12.00496103389468, segmentsPerTier=32.0, maxCFSSegmentSizeMB=1.078125, noCFSRatio=1.0
   [junit4]   2> 708810 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71de19d8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 708810 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 708811 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@f541914[collection1] main
   [junit4]   2> 708811 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 708814 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 708814 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 708814 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 708817 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 708820 INFO  (searcherExecutor-4046-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@f541914[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1520068095287230464
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1520068095287230464
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 708823 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 708824 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46663 collection:control_collection shard:shard1
   [junit4]   2> 708828 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 708833 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95004255776604164-core_node1-n_0000000000
   [junit4]   2> 708834 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46663/collection1/
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46663/collection1/ has no replicas
   [junit4]   2> 708837 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 708838 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 708839 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 708844 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 708846 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46663/collection1/ shard1
   [junit4]   2> 708848 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 708951 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 708998 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46663/collection1/ and leader is http://127.0.0.1:46663/collection1/
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46663
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 709004 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"} current state version: 3
   [junit4]   2> 709005 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"}
   [junit4]   2> 709111 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 709292 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709293 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709294 INFO  (zkCallback-1580-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f0fa4ae name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709294 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709294 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709294 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709300 INFO  (zkCallback-1581-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@347edc46 name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709300 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709301 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709306 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 709306 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 709422 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 709652 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores/collection1
   [junit4]   2> 709652 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001
   [junit4]   2> 709653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 709668 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@327a9510{/,null,AVAILABLE}
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d2fc2e1{HTTP/1.1}{127.0.0.1:60403}
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server Started @712507ms
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost:58696/hdfs__localhost_58696__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001_tempDir-002_jetty1, hostPort=60403, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores}
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001'
   [junit4]   2> 709670 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 709670 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 709670 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709676 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709676 INFO  (zkCallback-1582-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c94f01e name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709676 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709677 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709681 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 709681 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/solr.xml
   [junit4]   2> 709696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores
   [junit4]   2> 709696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer New CoreContainer 2146791045
   [junit4]   2> 709696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001]
   [junit4]   2> 709696 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/lib
   [junit4]   2> 709698 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34302/solr
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709703 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709707 INFO  (zkCallback-1584-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c11ac42 name:ZooKeeperConnection Watcher:127.0.0.1:34302 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709708 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709708 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709711 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15185de05b70008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 709716 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709718 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7dceea47 name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709718 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709728 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 710741 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60403_
   [junit4]   2> 710742 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60403_
   [junit4]   2> 710743 INFO  (zkCallback-1580-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 710743 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 710743 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 710746 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 710747 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95004255776604169-127.0.0.1:60403_-n_0000000001
   [junit4]   2> 710749 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95004255776604164-127.0.0.1:46663_-n_0000000000 to know if I could be the leader
   [junit4]   2> 710765 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 710765 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 710766 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores
   [junit4]   2> 710766 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {schema=schema.xml, loadOnStartup=true, coreNodeName=, shard=, collection=collection1, name=collection1, config=solrconfig.xml, transient=false, configSetProperties=configsetprops.json, dataDir=data/}
   [junit4]   2> 710767 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores/collection1
   [junit4]   2> 710767 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 710769 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 710769 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 710771 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 710771 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 710773 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 710773 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:60403_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60403"} current state version: 4
   [junit4]   2> 710773 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 710774 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:60403_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60403"}
   [junit4]   2> 710774 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 710774 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 710774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 710774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 710876 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 710877 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 710877 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 711774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 711774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 711775 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 711775 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 711776 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:coll

[...truncated too long message...]

terceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.teardownClass(HdfsBasicDistributedZk2Test.java:47)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 809787 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [n:127.0.0.1:59573_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001
   [junit4]   2> Dec 09, 2015 8:33:22 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {foo_b=FST50, n_l1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), intDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_d1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_tdt1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_i_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_ti1=PostingsFormat(name=Memory doPackFST= false), _version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), multiDefault=Lucene50(blocksize=128), id=PostingsFormat(name=Memory doPackFST= false), foo_f=FST50, n_tl1=Lucene50(blocksize=128), rnd_b=PostingsFormat(name=Memory doPackFST= false), range_facet_l=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), foo_d=PostingsFormat(name=Memory doPackFST= false), n_td1=Lucene50(blocksize=128), oddField_s=FST50, timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=FST50, SubjectTerms_mfacet=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_tf1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_f1=Lucene50(blocksize=128), n_dt1=Lucene50(blocksize=128), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= false), other_tl1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_i1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene54), timestamp=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=it_CH, timezone=Asia/Riyadh
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=5,free=268238792,total=466616320
   [junit4]   2> NOTE: All tests run in this JVM: [HighlighterMaxOffsetTest, TestSolrCoreProperties, TestSortByMinMaxFunction, SpellingQueryConverterTest, TestSurroundQueryParser, AnalyticsMergeStrategyTest, CollectionsAPIDistributedZkTest, TestStressVersions, SimplePostToolTest, ChaosMonkeySafeLeaderTest, TestJsonFacets, DistanceFunctionTest, TestRandomRequestDistribution, TestCloudPivotFacet, MoreLikeThisHandlerTest, TestRemoveLastDynamicCopyField, UpdateRequestProcessorFactoryTest, TestWordDelimiterFilterFactory, TestSolrConfigHandlerCloud, TestLRUStatsCache, TestSolrQueryParserDefaultOperatorResource, TestRawTransformer, TestHashQParserPlugin, HdfsThreadLeakTest, TestTrie, TestReplicationHandlerBackup, TestDistributedGrouping, TestLeaderElectionZkExpiry, TestDFRSimilarityFactory, BasicFunctionalityTest, TestMissingGroups, IndexBasedSpellCheckerTest, HdfsBasicDistributedZk2Test]
   [junit4] Completed [102/553 (1!)] on J1 in 115.89s, 1 test, 1 error <<< FAILURES!

[...truncated 1444 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 553 suites (7 ignored), 2188 tests, 1 error, 33 ignored (18 assumptions) [seed: BF6DB73668EA9BBA]

Total time: 203 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Got. I just didn't have jdk7 after last laptop crash. 5x build on jdk8
caught syntax errors, that's why I relaxed after fixing compile errors and
run tests. I didn't catch rt.jar difference. Now I'm able to test on jdk7.
Awaiting for CI.

On Tue, Dec 8, 2015 at 5:25 PM, david.w.smiley@gmail.com <
david.w.smiley@gmail.com> wrote:

> Mikhail,
> In the future, particularly when committing code that uses Java 8
> features, be sure to compile & run tests even on the 5x branch after
> back-porting but before committing.  From time to time this fails for
> stuff, like easy-to-forget cases of not using ‘final’.
> ~ David
>
> On Tue, Dec 8, 2015 at 2:21 AM Mikhail Khludnev <
> mkhludnev@griddynamics.com> wrote:
>
>> It's mine. Looking into.
>>
>> On Tue, Dec 8, 2015 at 9:35 AM, Apache Jenkins Server <
>> jenkins@builds.apache.org> wrote:
>>
>>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/
>>>
>>> All tests passed
>>>
>>> Build Log:
>>> [...truncated 7450 lines...]
>>>     [javac] Compiling 6 source files to
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
>>>     [javac]
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954:
>>> error: cannot find symbol
>>>     [javac]         assert nextInt ==
>>> Integer.parseUnsignedInt(uniqueRandomValue,16);
>>>     [javac]                                  ^
>>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>>     [javac]   location: class Integer
>>>     [javac]
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206:
>>> error: cannot find symbol
>>>     [javac]     final int linkInt =
>>> Integer.parseUnsignedInt(linkValue,16);
>>>     [javac]                                ^
>>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>>     [javac]   location: class Integer
>>>     [javac] Note:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java
>>> uses or overrides a deprecated API.
>>>     [javac] Note: Recompile with -Xlint:deprecation for details.
>>>     [javac] 2 errors
>>>
>>> BUILD FAILED
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968:
>>> Compile failed; see the compiler error output for details.
>>>
>>> Total time: 106 minutes 8 seconds
>>> Build step 'Invoke Ant' marked build as failure
>>> Archiving artifacts
>>> Recording test results
>>> Email was triggered for: Failure - Any
>>> Sending email for trigger: Failure - Any
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>> Principal Engineer,
>> Grid Dynamics
>>
>> <http://www.griddynamics.com>
>> <mk...@griddynamics.com>
>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mk...@griddynamics.com>

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

Posted by "david.w.smiley@gmail.com" <da...@gmail.com>.
Mikhail,
In the future, particularly when committing code that uses Java 8 features,
be sure to compile & run tests even on the 5x branch after back-porting but
before committing.  From time to time this fails for stuff, like
easy-to-forget cases of not using ‘final’.
~ David

On Tue, Dec 8, 2015 at 2:21 AM Mikhail Khludnev <mk...@griddynamics.com>
wrote:

> It's mine. Looking into.
>
> On Tue, Dec 8, 2015 at 9:35 AM, Apache Jenkins Server <
> jenkins@builds.apache.org> wrote:
>
>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/
>>
>> All tests passed
>>
>> Build Log:
>> [...truncated 7450 lines...]
>>     [javac] Compiling 6 source files to
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
>>     [javac]
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954:
>> error: cannot find symbol
>>     [javac]         assert nextInt ==
>> Integer.parseUnsignedInt(uniqueRandomValue,16);
>>     [javac]                                  ^
>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>     [javac]   location: class Integer
>>     [javac]
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206:
>> error: cannot find symbol
>>     [javac]     final int linkInt =
>> Integer.parseUnsignedInt(linkValue,16);
>>     [javac]                                ^
>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>     [javac]   location: class Integer
>>     [javac] Note:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java
>> uses or overrides a deprecated API.
>>     [javac] Note: Recompile with -Xlint:deprecation for details.
>>     [javac] 2 errors
>>
>> BUILD FAILED
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968:
>> Compile failed; see the compiler error output for details.
>>
>> Total time: 106 minutes 8 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <mk...@griddynamics.com>
>
-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

Posted by Mikhail Khludnev <mk...@griddynamics.com>.
It's mine. Looking into.

On Tue, Dec 8, 2015 at 9:35 AM, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/
>
> All tests passed
>
> Build Log:
> [...truncated 7450 lines...]
>     [javac] Compiling 6 source files to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
>     [javac]
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954:
> error: cannot find symbol
>     [javac]         assert nextInt ==
> Integer.parseUnsignedInt(uniqueRandomValue,16);
>     [javac]                                  ^
>     [javac]   symbol:   method parseUnsignedInt(String,int)
>     [javac]   location: class Integer
>     [javac]
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206:
> error: cannot find symbol
>     [javac]     final int linkInt = Integer.parseUnsignedInt(linkValue,16);
>     [javac]                                ^
>     [javac]   symbol:   method parseUnsignedInt(String,int)
>     [javac]   location: class Integer
>     [javac] Note:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java
> uses or overrides a deprecated API.
>     [javac] Note: Recompile with -Xlint:deprecation for details.
>     [javac] 2 errors
>
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968:
> Compile failed; see the compiler error output for details.
>
> Total time: 106 minutes 8 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mk...@griddynamics.com>

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/

All tests passed

Build Log:
[...truncated 7450 lines...]
    [javac] Compiling 6 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954: error: cannot find symbol
    [javac]         assert nextInt == Integer.parseUnsignedInt(uniqueRandomValue,16);
    [javac]                                  ^
    [javac]   symbol:   method parseUnsignedInt(String,int)
    [javac]   location: class Integer
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206: error: cannot find symbol
    [javac]     final int linkInt = Integer.parseUnsignedInt(linkValue,16);
    [javac]                                ^
    [javac]   symbol:   method parseUnsignedInt(String,int)
    [javac]   location: class Integer
    [javac] Note: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java uses or overrides a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 2 errors

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968: Compile failed; see the compiler error output for details.

Total time: 106 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1039 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1039/

3 tests failed.
FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testCollectionCreateSearchDelete

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974:C8B086780ED823D1]:0)
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
	at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
	at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.server.Server.doStart(Server.java:366)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:407)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:357)
	at org.apache.solr.cloud.TestMiniSolrCloudCluster.testCollectionCreateSearchDelete(TestMiniSolrCloudCluster.java:190)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestAuthenticationFramework

Error Message:
10 threads leaked from SUITE scope at org.apache.solr.cloud.TestAuthenticationFramework:     1) Thread[id=50657, name=qtp1598845069-50657, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=50656, name=qtp1598845069-50656, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=50655, name=qtp1598845069-50655, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=50658, name=qtp1598845069-50658, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=50654, name=qtp1598845069-50654, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=50660, name=qtp1598845069-50660, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=50653, name=qtp1598845069-50653, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=50659, name=qtp1598845069-50659, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.cloud.TestAuthenticationFramework: 
   1) Thread[id=50657, name=qtp1598845069-50657, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=50656, name=qtp1598845069-50656, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=50655, name=qtp1598845069-50655, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=50658, name=qtp1598845069-50658, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=50654, name=qtp1598845069-50654, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=50660, name=qtp1598845069-50660, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=50653, name=qtp1598845069-50653, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=50659, name=qtp1598845069-50659, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestAuthenticationFramework

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)




Build Log:
[...truncated 10595 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
   [junit4]   2> 855113 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 855114 INFO  (Thread-48265) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 855114 INFO  (Thread-48265) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 855214 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer start zk server on port:51098
   [junit4]   2> 855214 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855214 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855223 INFO  (zkCallback-386-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c315d15 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855223 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855223 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855224 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 855230 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 855230 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 855231 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 855234 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@689a6a7e{/solr,null,AVAILABLE}
   [junit4]   2> 855234 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37db9ed2{/solr,null,AVAILABLE}
   [junit4]   2> 855234 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@339809a4{/solr,null,AVAILABLE}
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d926c48{HTTP/1.1}{127.0.0.1:55066}
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.Server Started @858130ms
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=55066, hostContext=/solr}
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1'
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 855239 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855239 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855239 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@65f9f092{HTTP/1.1}{127.0.0.1:48544}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.Server Started @858132ms
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=48544, hostContext=/solr}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@c999844{HTTP/1.1}{127.0.0.1:55562}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.Server Started @858132ms
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=55562, hostContext=/solr}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2'
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3'
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855242 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855244 INFO  (zkCallback-387-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f2809d6 name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855244 INFO  (zkCallback-388-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@579a9cba name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855245 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 855250 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 855250 INFO  (zkCallback-389-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44c4ad4c name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855250 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855250 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855252 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 855267 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/.
   [junit4]   2> 855267 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer New CoreContainer 399278526
   [junit4]   2> 855267 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2]
   [junit4]   2> 855267 WARN  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/lib
   [junit4]   2> 855268 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 855268 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/.
   [junit4]   2> 855268 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 547657681
   [junit4]   2> 855269 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1]
   [junit4]   2> 855269 WARN  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/lib
   [junit4]   2> 855269 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 855271 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51098/solr
   [junit4]   2> 855271 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51098/solr
   [junit4]   2> 855271 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 855272 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 855272 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855272 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/.
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer New CoreContainer 54608992
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3]
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855273 WARN  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/lib
   [junit4]   2> 855274 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 855274 INFO  (zkCallback-393-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40e303e4 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855274 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 855274 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51098/solr
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855275 INFO  (zkCallback-392-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ea0ae38 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855280 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855280 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855280 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855281 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855282 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43833910 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855282 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855283 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855291 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855291 INFO  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@703a309f name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855291 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855291 INFO  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7073b9fc name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855291 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855292 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855294 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 855294 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 855294 INFO  (zkCallback-398-thread-1-processing-n:127.0.0.1:55562_solr) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@685b0835 name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855294 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855300 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 855300 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 855301 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 855307 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 855308 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 855308 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 855311 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 855311 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 855316 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 855316 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 855320 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 855320 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 855320 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 855325 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 855325 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 855325 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 855331 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 855331 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 855332 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 855334 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 855335 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 855335 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 855337 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 855337 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 855337 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 855339 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48544_solr
   [junit4]   2> 855339 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55562_solr
   [junit4]   2> 855340 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55066_solr
   [junit4]   2> 855340 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48544_solr
   [junit4]   2> 855341 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55562_solr
   [junit4]   2> 855341 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55066_solr
   [junit4]   2> 855343 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 855344 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 855346 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 855346 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 855346 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 855348 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 855348 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 855348 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94992915367526407-127.0.0.1:55066_solr-n_0000000000
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94992915367526409-127.0.0.1:55562_solr-n_0000000001
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94992915367526408-127.0.0.1:48544_solr-n_0000000002
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55066_solr
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94992915367526407-127.0.0.1:55066_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94992915367526409-127.0.0.1:55562_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855353 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Overseer (id=94992915367526407-127.0.0.1:55066_solr-n_0000000000) starting
   [junit4]   2> 855364 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 855364 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CoreContainer Authentication plugin class obtained from system property 'authenticationPlugin': org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 855365 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CoreContainer Reconfiguring the shard handler factory and update shard handler.
   [junit4]   2> 855365 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.h.c.HttpShardHandlerFactory Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@28ba9edb
   [junit4]   2> 855365 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.u.UpdateShardHandler Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@28ba9edb
   [junit4]   2> 855366 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/.
   [junit4]   2> 855366 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 855374 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 855374 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 855376 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 855381 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 855381 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CoreContainer Authentication plugin class obtained from system property 'authenticationPlugin': org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 855382 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CoreContainer Reconfiguring the shard handler factory and update shard handler.
   [junit4]   2> 855382 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.h.c.HttpShardHandlerFactory Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@404c9413
   [junit4]   2> 855382 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.u.UpdateShardHandler Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@404c9413
   [junit4]   2> 855383 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/.
   [junit4]   2> 855384 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 855389 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 855389 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 855394 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 855394 INFO  (OverseerCollectionConfigSetProcessor-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 855394 INFO  (OverseerStateUpdate-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 855395 INFO  (OverseerStateUpdate-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855404 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855430 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 855430 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CoreContainer Authentication plugin class obtained from system property 'authenticationPlugin': org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 855430 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CoreContainer Reconfiguring the shard handler factory and update shard handler.
   [junit4]   2> 855431 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.h.c.HttpShardHandlerFactory Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@40284289
   [junit4]   2> 855431 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.u.UpdateShardHandler Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@40284289
   [junit4]   2> 855432 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/.
   [junit4]   2> 855433 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 855433 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 855434 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 855434 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855434 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855436 INFO  (zkCallback-399-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c79c234 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855436 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855437 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855441 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@c999844{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 855441 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=54608992
   [junit4]   2> 855443 INFO  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 855444 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@6d926c48{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 855448 ERROR (jetty-launcher-385-thread-3-EventThread) [n:127.0.0.1:55562_solr    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@d1129e1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@61d4797a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:215)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 855450 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@689a6a7e{/solr,null,UNAVAILABLE}
   [junit4]   2> 855449 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=547657681
   [junit4]   2> 855448 INFO  (zkCallback-396-thread-2-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855451 INFO  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 855457 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@65f9f092{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 855460 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.Overseer Overseer (id=94992915367526407-127.0.0.1:55066_solr-n_0000000000) closing
   [junit4]   2> 855460 INFO  (OverseerStateUpdate-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55066_solr
   [junit4]   2> 855460 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=399278526
   [junit4]   2> 855464 INFO  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855464 WARN  (zkCallback-397-thread-2-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector Failed setting watch
   [junit4]   2> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer_elect/election/94992915367526407-127.0.0.1:55066_solr-n_0000000000
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:182)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:59)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:389)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 855464 INFO  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855465 INFO  (zkCallback-396-thread-2-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855465 WARN  (zkCallback-396-thread-2-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 856964 WARN  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 856965 WARN  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 856966 WARN  (zkCallback-397-thread-2-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector 
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:328)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:325)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:325)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:190)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:59)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:389)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 856966 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@37db9ed2{/solr,null,UNAVAILABLE}
   [junit4]   2> 856968 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@339809a4{/solr,null,UNAVAILABLE}
   [junit4]   2> 856972 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51098 51098
   [junit4]   2> 857194 INFO  (Thread-48265) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51098 51098
   [junit4]   2> 857195 WARN  (Thread-48265) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/security.json
   [junit4]   2> 	4	/solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 857202 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 857202 INFO  (Thread-48302) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 857202 INFO  (Thread-48302) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 857302 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer start zk server on port:45652
   [junit4]   2> 857302 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857303 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857310 INFO  (zkCallback-402-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1006d0ee name:ZooKeeperConnection Watcher:127.0.0.1:45652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857311 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857311 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857311 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 857320 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857320 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857320 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857322 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857322 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857324 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@50fcf69a{/solr,null,AVAILABLE}
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@4a64f75f{HTTP/1.1}{127.0.0.1:57569}
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.Server Started @860217ms
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57569}
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1'
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ea123a9{/solr,null,AVAILABLE}
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.ServerConnector Started ServerConnector@159a28e7{HTTP/1.1}{127.0.0.1:51048}
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.Server Started @860218ms
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=51048, hostContext=/solr}
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5'
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857328 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@8d2b986{/solr,null,AVAILABLE}
   [junit4]   2> 857329 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2875274a{/solr,null,AVAILABLE}
   [junit4]   2> 857329 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@3ef03973{HTTP/1.1}{127.0.0.1:49406}
   [junit4]   2> 857329 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.Server Started @860221ms
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49406}
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node2'
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857331 INFO  (zkCallback-404-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14f71064 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857331 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857331 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857331 INFO  (zkCallback-403-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d3420 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857331 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@3a3a0c92{HTTP/1.1}{127.0.0.1:57931}
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.Server Started @860225ms
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57931}
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node3'
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857334 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857336 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54d573b6{/solr,null,AVAILABLE}
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.ServerConnector Started ServerConnector@99fdd3d{HTTP/1.1}{127.0.0.1:44879}
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.Server Started @860229ms
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44879}
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node4'
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857338 INFO  (zkCallback-405-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@177e48ed name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857338 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857338 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857339 INFO  (zkCallback-406-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5abcee89 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857344 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857345 INFO  (zkCallback-407-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1beffb48 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857346 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857346 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857346 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857351 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1/.
   [junit4]   2> 857352 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5/.
   [junit4]   2> 857352 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CoreContainer New CoreContainer 844108746
   [junit4]   2> 857352 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5]
   [junit4]   2> 857352 WARN  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5/lib
   [junit4]   2> 857353 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 857354 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45652/solr
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 626263853
   [junit4]   2> 857356 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857360 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1]
   [junit4]   2> 857361 WARN  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Nigh

[...truncated too long message...]

0(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=50658, name=qtp1598845069-50658, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=50654, name=qtp1598845069-50654, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    7) Thread[id=50660, name=qtp1598845069-50660, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    8) Thread[id=50653, name=qtp1598845069-50653, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    9) Thread[id=50659, name=qtp1598845069-50659, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   10) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)
   [junit4] Completed [131/553] on J2 in 123.11s, 7 tests, 3 errors <<< FAILURES!

[...truncated 1295 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 553 suites (7 ignored), 2187 tests, 2 suite-level errors, 1 error, 1252 ignored (15 assumptions) [seed: 6B4A28DD8930C974]

Total time: 231 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1038 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1038/

2 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]> but was:<[{indexVersion=1449417291014,generation=3,filelist=[_6.cfe, _6.cfs, _6.si, segments_3]}, {indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]> but was:<[{indexVersion=1449417291014,generation=3,filelist=[_6.cfe, _6.cfs, _6.si, segments_3]}, {indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]>
	at __randomizedtesting.SeedInfo.seed([DA5873B6F3FA9AF8:FF8F688683B294FB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1140)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Mon Dec 07 02:56:23 SBT 2015

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Mon Dec 07 02:56:23 SBT 2015
	at __randomizedtesting.SeedInfo.seed([DA5873B6F3FA9AF8:1F37370F6D2F34B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1419)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:771)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10731 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/init-core-data-001
   [junit4]   2> 1225543 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestRepeater
   [junit4]   2> 1225544 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1
   [junit4]   2> 1225580 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1225637 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@108ae9d5{/solr,null,AVAILABLE}
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@55fb709c{HTTP/1.1}{127.0.0.1:34195}
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1228780ms
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34195, hostContext=/solr, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/data}
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001'
   [junit4]   2> 1225655 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1225655 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1225655 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/solr.xml
   [junit4]   2> 1225699 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/.
   [junit4]   2> 1225699 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1822242929
   [junit4]   2> 1225699 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001]
   [junit4]   2> 1225700 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/lib
   [junit4]   2> 1225701 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1225703 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1225704 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1225704 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1225705 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1225705 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1225707 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/.
   [junit4]   2> 1225715 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, transient=false, loadOnStartup=true, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, schema=schema.xml}
   [junit4]   2> 1225716 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1
   [junit4]   2> 1225716 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1225727 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1'
   [junit4]   2> 1225738 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1225744 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1225744 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1225744 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1225753 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1225766 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1225790 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1225795 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1225814 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1225817 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1'
   [junit4]   2> 1225817 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1
   [junit4]   2> 1225817 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1225818 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1], dataDir=[null]
   [junit4]   2> 1225818 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1225818 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data
   [junit4]   2> 1225819 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 1225819 WARN  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1225819 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 1225819 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1225820 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_1,generation=1}
   [junit4]   2> 1225820 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1225845 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1225848 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225849 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225850 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1225851 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225852 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225858 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1225858 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1225863 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1225864 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1225864 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1225866 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@790ec6e7
   [junit4]   2> 1225866 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_1,generation=1}
   [junit4]   2> 1225866 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@15000ac7[collection1] main
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/conf
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/conf
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1225868 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1225868 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1225868 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1225880 INFO  (searcherExecutor-4440-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@15000ac7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1226246 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1
   [junit4]   2> 1226249 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1226271 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a046505{/solr,null,AVAILABLE}
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@26843066{HTTP/1.1}{127.0.0.1:39777}
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1229398ms
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/data, hostPort=39777, hostContext=/solr}
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002'
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226273 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/solr.xml
   [junit4]   2> 1226317 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226318 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 443998018
   [junit4]   2> 1226318 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002]
   [junit4]   2> 1226318 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib
   [junit4]   2> 1226321 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1226326 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1226328 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226338 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, dataDir=data/, loadOnStartup=true, transient=false, configSetProperties=configsetprops.json, config=solrconfig.xml, schema=schema.xml}
   [junit4]   2> 1226339 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226339 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1226358 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226359 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1226359 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1226359 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226359 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226367 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1226494 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1226584 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1226600 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1226617 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1226621 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226621 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226621 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1226622 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1], dataDir=[null]
   [junit4]   2> 1226622 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1226625 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data
   [junit4]   2> 1226625 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1226625 WARN  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1226626 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1226626 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1226629 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@19dd431f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d61b57d),segFN=segments_1,generation=1}
   [junit4]   2> 1226629 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226643 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1226645 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226646 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226647 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226648 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226649 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1226659 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1226659 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1226660 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1226667 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1226667 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1226667 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4746c9ee
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@19dd431f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d61b57d),segFN=segments_1,generation=1}
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@59b82977[collection1] main
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226669 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1226671 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1226671 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1226672 INFO  (searcherExecutor-4450-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59b82977[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1226866 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@26843066{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 1226866 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=443998018
   [junit4]   2> 1226866 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@661bb296
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index;done=false>>]
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data;done=false>>]
   [junit4]   2> 1226871 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data
   [junit4]   2> 1226872 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@a046505{/solr,null,UNAVAILABLE}
   [junit4]   2> 1226874 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1226877 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@b6329e9{/solr,null,AVAILABLE}
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1b5485d6{HTTP/1.1}{127.0.0.1:32879}
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1230004ms
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=32879, hostContext=/solr, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/data}
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002'
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226879 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226879 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/solr.xml
   [junit4]   2> 1226890 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226890 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1045830494
   [junit4]   2> 1226890 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002]
   [junit4]   2> 1226891 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib
   [junit4]   2> 1226892 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1226894 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226895 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, config=solrconfig.xml, dataDir=data/, name=collection1, configSetProperties=configsetprops.json, schema=schema.xml, loadOnStartup=true}
   [junit4]   2> 1226895 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226895 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1226895 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226897 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226897 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226905 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1226905 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1226907 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1226920 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1226944 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1226949 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1226956 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1226958 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226958 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226959 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1226959 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1], dataDir=[null]
   [junit4]   2> 1226959 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1226960 WARN  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1226961 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@245d75ca lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73e0336c),segFN=segments_1,generation=1}
   [junit4]   2> 1226961 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226963 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1226966 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1226966 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226967 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226969 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226970 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226971 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1226971 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1226972 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1226973 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1226973 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1226973 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7d1798fb
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@245d75ca lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73e0336c),segFN=segments_1,generation=1}
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3e263278[collection1] main
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1226975 INFO  (searcherExecutor-4461-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3e263278[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1227406 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1
   [junit4]   2> 1227411 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34d9a745{/solr,null,AVAILABLE}
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3d74be1e{HTTP/1.1}{127.0.0.1:49434}
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1230544ms
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=49434}
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003'
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/solr.xml
   [junit4]   2> 1227431 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/.
   [junit4]   2> 1227431 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1316711039
   [junit4]   2> 1227431 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003]
   [junit4]   2> 1227431 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/lib
   [junit4]   2> 1227433 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1227435 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/.
   [junit4]   2> 1227436 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, loadOnStartup=true, transient=false, schema=schema.xml, configSetProperties=configsetprops.json, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 1227436 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1
   [junit4]   2> 1227436 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1227437 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1'
   [junit4]   2> 1227444 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1227444 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1227444 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1227444 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1227454 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1227475 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1227517 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1227521 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1227528 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1'
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1], dataDir=[null]
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1227531 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data
   [junit4]   2> 1227531 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index/
   [junit4]   2> 1227532 WARN  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227534 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/
   [junit4]   2> 1227536 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1227538 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227539 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227540 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227541 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227542 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1227543 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6f0d3507
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5be8056f[collection1] main
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/conf
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/conf}
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/conf
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1227546 WARN  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler  No value set for 'pollInterval'. Timer Task not started.
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1227547 INFO  (searcherExecutor-4471-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5be8056f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1227950 INFO  (qtp755704053-11237) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 0
   [junit4]   2> 1227953 INFO  (qtp755704053-11236) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0
   [junit4]   2> 1227955 INFO  (qtp755704053-11238) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 0
   [junit4]   2> 1227957 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1227959 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_2,generation=2}
   [junit4]   2> 1227960 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 1227962 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2556203e[collection1] main
   [junit4]   2> 1227962 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1227962 INFO  (searcherExecutor-4440-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2556203e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C3)))}
   [junit4]   2> 1227962 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&commit=true&wt=javabin&version=2} {commit=} 0 4
   [junit4]   2> 1227966 WARN  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 1227972 INFO  (qtp755704053-11240) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 1227972 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 1227972 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 1227972 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 1227977 INFO  (qtp755704053-11237) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=filelist&qt=/replication&generation=2&wt=javabin&version=2} status=0 QTime=2 
   [junit4]   2> 1227977 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 10
   [junit4]   2> 1227980 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index.20151207024828861
   [junit4]   2> 1227980 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.DefaultSolrCoreState Rollback old IndexWriter... core=collection1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.DefaultSolrCoreState Rollback old IndexWriter... core=collection1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/index-NIOFSDirectory-005 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a3dbf04; maxCacheMB=0.0126953125 maxMergeSizeMB=0.09375))
   [junit4]   2> 1227995 INFO  (qtp755704053-11236) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={checksum=true&file=_0.fdx&command=filecontent&qt=/replication&generation=2&wt=filestream} status=0 QTime=3 
   [junit4]   2> 1228000 INFO  (qtp755704053-11238) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={checksum=true&file=_0.nvd&command=filecontent&qt=/replication&generation=2&wt=filestream} status=0 QTime=0 
   [junit4]   2> 1228004 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={checksum=true&file=_0_Lucene50_0.doc&command=filecontent&qt=/replication&generation=2&wt=filestream} status=0 QTime=0 
   [junit4]   2> 1228007 INFO  (qtp755704053

[...truncated too long message...]

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.1.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151206_152757_655.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151206_152757_655.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 221 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1037 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1037/

2 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestGeoPointQuery

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)




Build Log:
[...truncated 8703 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> ??? 05, 2015 1:58:23 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2>    1) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:601)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:450)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2>    2) Thread[id=38, name=T2, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Long.valueOf(Long.java:577)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.mortonHash(GeoUtils.java:50)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.radiusQueryCanBeWrong(TestGeoPointQuery.java:206)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.distanceRangeContainsPoint(TestGeoPointQuery.java:192)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2$2.shouldMatch(BaseGeoPointTestCase.java:701)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:526)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    3) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2>    4) Thread[id=16, name=SUITE-TestGeoPointQuery-seed#[9F451249890883AF], state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:591)
   [junit4]   2>    5) Thread[id=17, name=TEST-TestGeoPointQuery.testRandomBig-seed#[9F451249890883AF], state=WAITING, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:764)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    6) Thread[id=36, name=T0, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    7) Thread[id=37, name=T1, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    8) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:109)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.<init>(GeoPointDistanceQueryImpl.java:58)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl.getTermsEnum(GeoPointDistanceQueryImpl.java:47)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:198)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:239)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=9F451249890883AF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el_CY -Dtests.timezone=America/Argentina/Jujuy -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   7416s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)
   [junit4]   2> ??? 05, 2015 2:18:43 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 5 leaked thread(s).
   [junit4]   2> ??? 05, 2015 2:28:02 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 5 threads leaked from SUITE scope at org.apache.lucene.search.TestGeoPointQuery: 
   [junit4]   2>    1) Thread[id=17, name=TEST-TestGeoPointQuery.testRandomBig-seed#[9F451249890883AF], state=WAITING, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:764)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    2) Thread[id=36, name=T0, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    3) Thread[id=37, name=T1, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    4) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:109)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.<init>(GeoPointDistanceQueryImpl.java:58)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl.getTermsEnum(GeoPointDistanceQueryImpl.java:47)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:198)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:239)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    5) Thread[id=38, name=T2, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Long.valueOf(Long.java:577)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.mortonHash(GeoUtils.java:50)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.radiusQueryCanBeWrong(TestGeoPointQuery.java:203)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.distanceRangeContainsPoint(TestGeoPointQuery.java:192)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2$2.shouldMatch(BaseGeoPointTestCase.java:701)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:526)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2> ??? 05, 2015 2:30:42 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=17, name=TEST-TestGeoPointQuery.testRandomBig-seed#[9F451249890883AF], state=WAITING, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    2) Thread[id=36, name=T0, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    3) Thread[id=37, name=T1, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    4) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    5) Thread[id=38, name=T2, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2> ??? 05, 2015 2:36:12 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=38, name=T2, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Integer.valueOf(Integer.java:642)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:521)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    2) Thread[id=17, name=SUITE-TestGeoPointQuery-seed#[9F451249890883AF]-worker, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$1.next(Collections.java:1403)
   [junit4]   2>         at java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$1.next(Collections.java:1396)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ClassModel.getAnnotatedLeafMethods(ClassModel.java:254)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.getShuffledMethods(RandomizedRunner.java:1254)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$500(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:832)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=36, name=T0, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellCrosses(GeoPointInPolygonQuery.java:150)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellIntersectsShape(GeoPointInPolygonQuery.java:162)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:105)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    4) Thread[id=37, name=T1, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:111)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    5) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.isClosestPointOnRectWithinRange(GeoUtils.java:368)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesCircle(GeoUtils.java:362)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.cellCrosses(GeoPointDistanceQueryImpl.java:80)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.cellIntersectsShape(GeoPointDistanceQueryImpl.java:90)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:105)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.<init>(GeoPointDistanceQueryImpl.java:58)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl.getTermsEnum(GeoPointDistanceQueryImpl.java:47)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:198)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:239)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {id=FSTOrd50, point=PostingsFormat(name=Direct)}, docValues:{id=DocValuesFormat(name=Memory), point=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=el_CY, timezone=America/Argentina/Jujuy
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=6,free=38789464,total=463470592
   [junit4]   2> NOTE: All tests run in this JVM: [DuplicateFilterTest, TestGeoUtils, TestGeoPointQuery]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.seed=9F451249890883AF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el_CY -Dtests.timezone=America/Argentina/Jujuy -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J1 | TestGeoPointQuery (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)
   [junit4] Completed [20/20] on J1 in 10245.45s, 23 tests, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 20 suites, 142 tests, 1 suite-level error, 1 error, 1 ignored (1 assumption) [seed: 9F451249890883AF]

Total time: 291 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1036 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1036/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=32047, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=32047, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:48527/kn_u: Could not find collection : awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([42AE944646D3DDD0]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([42AE944646D3DDD0:9C24FA3B7D71BF23]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:175)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:136)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:831)
	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10717 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/init-core-data-001
   [junit4]   2> 1001602 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[42AE944646D3DDD0]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1001604 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[42AE944646D3DDD0]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /kn_u/
   [junit4]   2> 1001618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1001618 INFO  (Thread-25889) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1001618 INFO  (Thread-25889) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1001718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkTestServer start zk server on port:39818
   [junit4]   2> 1001719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1001720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1001724 INFO  (zkCallback-1800-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1878258a name:ZooKeeperConnection Watcher:127.0.0.1:39818 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1001724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1001724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1001724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1001728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1001732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1001732 INFO  (zkCallback-1801-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53db2de9 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1001732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1001733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1001733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1001735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1001737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1001739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1001741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1001741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1001743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1001744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1001745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1001746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1001747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1001747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1001749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1001749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1001751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1001751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1001753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1001754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1001757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1001757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1001758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1001758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1001760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1001760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1001761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1001761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1002076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1
   [junit4]   2> 1002077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1002081 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2dff72ad{/kn_u,null,AVAILABLE}
   [junit4]   2> 1002083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5dc1d372{HTTP/1.1}{127.0.0.1:48527}
   [junit4]   2> 1002083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server Started @1005020ms
   [junit4]   2> 1002083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=48527, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/tempDir-001/control/data, hostContext=/kn_u, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores}
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001'
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1002086 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1002088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002088 INFO  (zkCallback-1802-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70b6f9a8 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1002089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1002090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1002090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/solr.xml
   [junit4]   2> 1002105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores
   [junit4]   2> 1002105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer New CoreContainer 1194329061
   [junit4]   2> 1002105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001]
   [junit4]   2> 1002106 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/lib
   [junit4]   2> 1002107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1002112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39818/solr
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1002115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002116 INFO  (zkCallback-1804-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@567d5ff0 name:ZooKeeperConnection Watcher:127.0.0.1:39818 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1002117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1002119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002120 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57fe7b3b name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1002122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1002124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1002127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1002129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1002132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1002134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1002135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1002137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1002138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1002139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48527_kn_u
   [junit4]   2> 1002140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48527_kn_u
   [junit4]   2> 1002141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1002143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1002144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1002145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94978216905998340-127.0.0.1:48527_kn_u-n_0000000000
   [junit4]   2> 1002145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48527_kn_u
   [junit4]   2> 1002145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1002146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer Overseer (id=94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) starting
   [junit4]   2> 1002148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1002155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1002156 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1002156 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1002167 INFO  (OverseerCollectionConfigSetProcessor-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1002170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1002199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1002199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1002200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores
   [junit4]   2> 1002201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=control_collection, coreNodeName=, schema=schema.xml, shard=, name=collection1, configSetProperties=configsetprops.json, dataDir=data/, loadOnStartup=true, config=solrconfig.xml, transient=false}
   [junit4]   2> 1002201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1
   [junit4]   2> 1002201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1002202 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1002202 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1002202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1002202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1002205 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 1002205 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1002206 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null}
   [junit4]   2> 1002206 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1002206 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1002208 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1002209 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003206 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1003206 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1003207 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1003208 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1'
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1003211 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1003221 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1003234 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1003278 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1003287 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1003532 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1003540 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1003542 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1003553 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1003556 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1003558 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1003559 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1003559 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1003559 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1003560 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1003560 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1003561 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1003561 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1003561 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@91d39bf
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1003563 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1003563 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/index
   [junit4]   2> 1003563 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=81.46875, floorSegmentMB=1.2275390625, forceMergeDeletesPctAllowed=27.03868587713784, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1003564 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fe7f867 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@566573cb),segFN=segments_1,generation=1}
   [junit4]   2> 1003564 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1003564 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/
   [junit4]   2> 1003564 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1003570 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1003570 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1003570 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1003573 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003574 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003575 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003576 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003580 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/threads,/update/json,/admin/luke,/update/json/docs,/update,/admin/system,/dump,/admin/segments,/admin/logging,/replication,/admin/plugins,/schema,standard,/admin/properties,/config,/admin/ping,/admin/file,/get,/update/csv
   [junit4]   2> 1003581 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1003583 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1273988799, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1003583 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fe7f867 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@566573cb),segFN=segments_1,generation=1}
   [junit4]   2> 1003583 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1003584 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1511f65d[collection1] main
   [junit4]   2> 1003584 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1003586 INFO  (searcherExecutor-2584-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1511f65d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1519651472558522368
   [junit4]   2> 1003589 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1519651472558522368
   [junit4]   2> 1003589 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1003590 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1003590 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:48527/kn_u collection:control_collection shard:shard1
   [junit4]   2> 1003591 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1003597 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94978216905998340-core_node1-n_0000000000
   [junit4]   2> 1003597 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48527/kn_u/collection1/
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:48527/kn_u/collection1/ has no replicas
   [junit4]   2> 1003600 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1003601 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1003604 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1003606 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48527/kn_u/collection1/ shard1
   [junit4]   2> 1003606 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1003705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1003705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1003707 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4be2b750 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1003707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1003708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1003708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1003709 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003709 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003709 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 1003712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:48527/kn_u/collection1/ and leader is http://127.0.0.1:48527/kn_u/collection1/
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48527/kn_u
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1003759 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1"} current state version: 2
   [junit4]   2> 1003760 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 1003863 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003863 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003863 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1004006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores/collection1
   [junit4]   2> 1004007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001
   [junit4]   2> 1004008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1004010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2f47336d{/kn_u,null,AVAILABLE}
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@770422d9{HTTP/1.1}{127.0.0.1:34048}
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server Started @1006948ms
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34048, solrconfig=solrconfig.xml, hostContext=/kn_u, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores}
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001'
   [junit4]   2> 1004012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1004012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1004012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1004013 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004014 INFO  (zkCallback-1808-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@371fbf99 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1004014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1004015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1004015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/solr.xml
   [junit4]   2> 1004029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores
   [junit4]   2> 1004030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer New CoreContainer 456153429
   [junit4]   2> 1004030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001]
   [junit4]   2> 1004030 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/lib
   [junit4]   2> 1004030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39818/solr
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1004033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004034 INFO  (zkCallback-1810-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4411ef82 name:ZooKeeperConnection Watcher:127.0.0.1:39818 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1004036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1004039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004040 INFO  (zkCallback-1811-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ec1a51 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004040 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1004045 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1005050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34048_kn_u
   [junit4]   2> 1005051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34048_kn_u
   [junit4]   2> 1005052 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005052 INFO  (zkCallback-1811-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005053 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005052 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005053 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1005055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94978216905998344-127.0.0.1:34048_kn_u-n_0000000001
   [junit4]   2> 1005056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94978216905998340-127.0.0.1:48527_kn_u-n_0000000000 to know if I could be the leader
   [junit4]   2> 1005066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1005066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1005067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores
   [junit4]   2> 1005068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=collection1, coreNodeName=, configSetProperties=configsetprops.json, dataDir=data/, transient=false, schema=schema.xml, name=collection1, shard=, loadOnStartup=true, config=solrconfig.xml}
   [junit4]   2> 1005068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores/collection1
   [junit4]   2> 1005068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1005069 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1005069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1005070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1005069 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34048_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:34048/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null} current state version: 3
   [junit4]   2> 1005072 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34048_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:34048/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null}
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1005175 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1005175 INFO  (zkCallback-1811-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1005175 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1005175 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1006073 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1006073 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1006074 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1006074 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1006075 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1006075 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores/collection1'
   [junit4]   2> 1006075 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1006076 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1006078 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1006086 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1006099 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1006127 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1006136 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1006392 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1006400 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1006402 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1006412 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1006417 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1006419 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1006420 WARN  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.OpenExchang

[...truncated too long message...]

9
   [junit4]   2> 3548190 WARN  (Thread-47272) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/aliases.json
   [junit4]   2> 	10	/solr/security.json
   [junit4]   2> 	10	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	17	/solr/collections/collection1/state.json
   [junit4]   2> 	11	/solr/clusterstate.json
   [junit4]   2> 	9	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	6	/solr/overseer_elect/election/94978356697956356-127.0.0.1:34738_zhis%2Fgm-n_0000000000
   [junit4]   2> 	2	/solr/overseer_elect/election/94978356697956377-127.0.0.1:50715_zhis%2Fgm-n_0000000007
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94978356697956377-core_node3-n_0000000006
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	11	/solr/collections
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=42AE944646D3DDD0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ca -Dtests.timezone=US/Mountain -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  413s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([42AE944646D3DDD0:9C24FA3B7D71BF23]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:175)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:136)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:831)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:104)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3548193 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[42AE944646D3DDD0]-worker) [n:127.0.0.1:50715_zhis%2Fgm c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_42AE944646D3DDD0-001
   [junit4]   2> Dec 04, 2015 6:52:03 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=28343, maxDocsPerChunk=9, blockSize=516), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=28343, blockSize=516)), sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ca, timezone=US/Mountain
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=109854896,total=525860864
   [junit4]   2> NOTE: All tests run in this JVM: [TestStressRecovery, RegexBoostProcessorTest, TestSolrIndexConfig, TestConfigSetsAPIZkFailure, DeleteReplicaTest, SolrCoreCheckLockOnStartupTest, TestCoreDiscovery, SecurityConfHandlerTest, TestFastWriter, TestCSVResponseWriter, TestDistribIDF, HdfsBasicDistributedZkTest, SuggesterWFSTTest, DistributedFacetPivotLongTailTest, TestReloadDeadlock, CurrencyFieldXmlFileTest, RecoveryZkTest, BadIndexSchemaTest, NoCacheHeaderTest, StatelessScriptUpdateProcessorFactoryTest, LukeRequestHandlerTest, QueryParsingTest, TestHighlightDedupGrouping, TestExtendedDismaxParser, TestJettySolrRunner, TestReload, TestManagedStopFilterFactory, LeaderElectionTest, VersionInfoTest, DistribCursorPagingTest, TestSearchPerf, DocExpirationUpdateProcessorFactoryTest, SearchHandlerTest, OverseerRolesTest, TestExactStatsCache, TestNonDefinedSimilarityFactory, TestDistributedStatsComponentCardinality, HdfsThreadLeakTest, DistanceUnitsTest, TestDistributedMissingSort, TestCollapseQParserPlugin, DistribDocExpirationUpdateProcessorTest, RankQueryTest, TestCloudManagedSchema, TestPostingsSolrHighlighter, TestIndexSearcher, BigEndianAscendingWordSerializerTest, XmlUpdateRequestHandlerTest, TestManagedSchemaFieldTypeResource, TestZkChroot, TestRuleBasedAuthorizationPlugin, TestStandardQParsers, RecoveryAfterSoftCommitTest, TestSolrDeletionPolicy1, FacetPivotSmallTest, TestUtils, TestOverriddenPrefixQueryForCustomFieldType, RollingRestartTest, TestWriterPerf, TestRestManager, TestSha256AuthenticationProvider, SuggesterTest, SolrXmlInZkTest, BadComponentTest, SaslZkACLProviderTest, TestGroupingSearch, TestPHPSerializedResponseWriter, TestXIncludeConfig, TestReplicationHandlerBackup, TestSolrQueryParserResource, BJQParserTest, TestRecovery, BufferStoreTest, PreAnalyzedUpdateProcessorTest, FullSolrCloudDistribCmdsTest, TestFreeTextSuggestions, TestRequestForwarding, TestExpandComponent, QueryResultKeyTest, SampleTest, RequestLoggingTest, DirectUpdateHandlerOptimizeTest, OverseerTest, CoreAdminHandlerTest, ClusterStateTest, TestIndexingPerformance, TestJmxMonitoredMap, HighlighterMaxOffsetTest, TestClusterStateMutator, AsyncCallRequestStatusResponseTest, ShardRoutingTest, BasicZkTest, TestReplicationHandler, AutoCommitTest, TestFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestStressReorder, TestStressVersions, DistributedTermsComponentTest, SolrCoreTest, PeerSyncTest, ConvertedLegacyTest, TestSort, TestFiltering, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, SoftAutoCommitTest, SolrIndexSplitterTest, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTSTTest, WordBreakSolrSpellCheckerTest, FieldMutatingUpdateProcessorTest, TestRemoteStreaming, DebugComponentTest, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, TermsComponentTest, TermVectorComponentTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, LoggingHandlerTest, TestCollationField, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, TestBinaryResponseWriter, HighlighterConfigTest, UpdateParamsTest, AlternateDirectoryTest, TestQuerySenderNoQuery, SolrIndexConfigTest, TestSolrDeletionPolicy2, MultiTermTest, TestDocSet, TestBinaryField, NumericFieldsTest, TestElisionMultitermQuery, TestConfig, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestSolrCoreProperties, DirectSolrConnectionTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, ResourceLoaderTest, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, PluginInfoTest, TestFastLRUCache, DateMathParserTest, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, RAMDirectoryFactoryTest, TestSolrJ, ZkNodePropsTest, SliceStateTest, FileUtilsTest, CircularListTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, AsyncMigrateRouteKeyTest, ConfigSetsAPITest, DeleteShardTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [439/552] on J2 in 413.76s, 1 test, 1 failure <<< FAILURES!

[...truncated 359 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 552 suites (7 ignored), 2185 tests, 1 error, 1 failure, 33 ignored (18 assumptions) [seed: 42AE944646D3DDD0]

Total time: 209 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1035 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1035/

2 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":53477; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":53477; 
	at __randomizedtesting.SeedInfo.seed([562707F3EC0A0F3D:5875B3FDED9C3D32]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy109.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":50361; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":50361; 
	at __randomizedtesting.SeedInfo.seed([562707F3EC0A0F3D:DE73382942F662C5]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy111.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy112.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:402)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)




Build Log:
[...truncated 18504 lines...]
   [junit4] Suite: org.apache.solr.hadoop.MorphlineBasicMiniMRTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1027 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1126 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 1168 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 1246 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.c.C.deprecation hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
   [junit4]   2> 1246 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 1246 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 1253 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 1256 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 ??? 03 23:01:28
   [junit4]   2> 1260 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 1261 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1263 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 1263 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 1293 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 1304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 1304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 1305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 1305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 1312 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 1780 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 1799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 1799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 1799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 1802 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 1802 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 1820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 1820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 1820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 1924 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImage Allocated new BlockPoolId: BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 1943 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1 has been successfully formatted.
   [junit4]   2> 1947 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name2 has been successfully formatted.
   [junit4]   2> 2187 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNStorageRetentionManager Going to retain 1 images with txid >= 0
   [junit4]   2> 2192 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode createNameNode []
   [junit4]   2> 2251 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2352 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl Scheduled snapshot period at 10 second(s).
   [junit4]   2> 2353 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl NameNode metrics system started
   [junit4]   2> 2356 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode fs.defaultFS is hdfs://127.0.0.1:0
   [junit4]   2> 2432 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.DFSUtil Starting Web-server for hdfs at: http://localhost:0
   [junit4]   2> 2599 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 2630 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2669 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 2673 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
   [junit4]   2> 2673 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 2674 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 2727 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
   [junit4]   2> 2730 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 2778 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 35720
   [junit4]   2> 2778 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2859 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_35720_hdfs____.cdvm0z/webapp
   [junit4]   2> 3151 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 5362 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35720
   [junit4]   2> 5378 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 5378 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 5379 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 5379 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 5380 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 5380 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 ??? 03 23:01:32
   [junit4]   2> 5380 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 5381 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5381 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 5381 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 5382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 5382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 5382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 5384 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 5384 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 5384 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 5385 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 5385 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 5392 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 5392 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 5393 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 5403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 5410 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name2/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 5414 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1/current
   [junit4]   2> 5415 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name2/current
   [junit4]   2> 5415 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImage No edit log streams selected.
   [junit4]   2> 5452 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImageFormatPBINode Loading 1 INodes.
   [junit4]   2> 5467 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImageFormatProtobuf Loaded FSImage in 0 seconds.
   [junit4]   2> 5467 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImage Loaded image for txid 0 from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1/current/fsimage_0000000000000000000
   [junit4]   2> 5474 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
   [junit4]   2> 5475 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSEditLog Starting log segment at 1
   [junit4]   2> 5549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameCache initialized with 0 entries 0 lookups
   [junit4]   2> 5549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Finished loading FSImage in 156 msecs
   [junit4]   2> 5993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode RPC server is binding to localhost:0
   [junit4]   2> 6008 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 6035 INFO  (Socket Reader #1 for port 43259) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 43259
   [junit4]   2> 6092 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode Clients are to use localhost:43259 to access this namenode/service.
   [junit4]   2> 6095 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Registered FSNamesystemState MBean
   [junit4]   2> 6123 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 6123 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 6123 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem initializing replication queues
   [junit4]   2> 6124 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.StateChange STATE* Leaving safe mode after 0 secs
   [junit4]   2> 6125 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.StateChange STATE* Network topology has 0 racks and 0 datanodes
   [junit4]   2> 6125 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.StateChange STATE* UnderReplicatedBlocks has 0 blocks
   [junit4]   2> 6131 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Total number of blocks            = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of invalid blocks          = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of under-replicated blocks = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of  over-replicated blocks = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of blocks being written    = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.StateChange STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 6 msec
   [junit4]   2> 6222 INFO  (IPC Server listener on 43259) [    ] o.a.h.i.Server IPC Server listener on 43259: starting
   [junit4]   2> 6222 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 6231 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode NameNode RPC up at: localhost/127.0.0.1:43259
   [junit4]   2> 6231 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Starting services required for active state
   [junit4]   2> 6255 INFO  (CacheReplicationMonitor(676048482)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Starting CacheReplicationMonitor with interval 30000 milliseconds
   [junit4]   2> 6256 INFO  (CacheReplicationMonitor(676048482)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Rescanning after 17636790587 milliseconds
   [junit4]   2> 6256 INFO  (CacheReplicationMonitor(676048482)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Scanned 0 directive(s) and 0 block(s) in 0 millisecond(s).
   [junit4]   2> 6258 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2
   [junit4]   2> 6344 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 6346 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 6354 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 6364 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:34131
   [junit4]   2> 6368 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 6368 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 6383 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 6385 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 6386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 6386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 6386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 6393 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 6394 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 56245
   [junit4]   2> 6394 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 6417 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_56245_datanode____.mzamnb/webapp
   [junit4]   2> 6604 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 7108 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56245
   [junit4]   2> 7110 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 7110 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 7145 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 7146 INFO  (Socket Reader #1 for port 34967) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 34967
   [junit4]   2> 7163 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:34967
   [junit4]   2> 7177 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 7181 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 7209 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:43259 starting to offer service
   [junit4]   2> 7225 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 7234 INFO  (IPC Server listener on 34967) [    ] o.a.h.i.Server IPC Server listener on 34967: starting
   [junit4]   2> 7239 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4
   [junit4]   2> 7397 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 7398 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 7398 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 7399 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:34571
   [junit4]   2> 7399 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 7399 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 7401 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 7403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 7411 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 7411 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 7411 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 7413 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 7413 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 51484
   [junit4]   2> 7414 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 7468 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_51484_datanode____.uist8l/webapp
   [junit4]   2> 7774 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 7994 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 7998 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8001 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8001 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8008 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8009 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8009 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8168 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8168 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8170 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8170 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8171 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8174 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8175 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8175 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8175 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8178 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8182 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51484
   [junit4]   2> 8185 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8185 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 8185 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 8185 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 8187 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1179921729;bpid=BP-1268983991-10.41.0.5-1449176489074;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1179921729;c=0;bpid=BP-1268983991-10.41.0.5-1449176489074;dnuuid=null
   [junit4]   2> 8190 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID a1a5894e-b5bd-430b-b6b0-af8a7bb588de
   [junit4]   2> 8191 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:49663
   [junit4]   2> 8200 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 8201 INFO  (Socket Reader #1 for port 49663) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 49663
   [junit4]   2> 8213 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 8217 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:43259 starting to offer service
   [junit4]   2> 8217 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 8218 INFO  (IPC Server listener on 49663) [    ] o.a.h.i.Server IPC Server listener on 49663: starting
   [junit4]   2> 8230 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 8234 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8235 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8235 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8240 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8250 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8250 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8309 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current
   [junit4]   2> 8341 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current, StorageType: DISK
   [junit4]   2> 8342 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current
   [junit4]   2> 8342 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current, StorageType: DISK
   [junit4]   2> 8391 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Registered FSDatasetState MBean
   [junit4]   2> 8402 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DirectoryScanner Periodic Directory Tree Verification scan starting at 1449189806630 with interval 21600000
   [junit4]   2> 8403 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Adding block pool BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8403 INFO  (Thread-89) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Scanning block pool BP-1268983991-10.41.0.5-1449176489074 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current...
   [junit4]   2> 8404 INFO  (Thread-90) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Scanning block pool BP-1268983991-10.41.0.5-1449176489074 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current...
   [junit4]   2> 8516 INFO  (Thread-89) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time taken to scan block pool BP-1268983991-10.41.0.5-1449176489074 on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current: 106ms
   [junit4]   2> 8516 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8517 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8518 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8518 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8518 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8521 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8522 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8522 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8522 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8525 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8525 INFO  (Thread-90) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time taken to scan block pool BP-1268983991-10.41.0.5-1449176489074 on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current: 96ms
   [junit4]   2> 8525 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Total time to scan all replicas for block pool BP-1268983991-10.41.0.5-1449176489074: 122ms
   [junit4]   2> 8525 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8537 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1179921729;bpid=BP-1268983991-10.41.0.5-1449176489074;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1179921729;c=0;bpid=BP-1268983991-10.41.0.5-1449176489074;dnuuid=null
   [junit4]   2> 8541 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 59e613d4-abae-4102-9e3d-5075dd9f97bc
   [junit4]   2> 8542 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkin

[...truncated too long message...]

r:core_node5 x:collection1] o.a.h.h.s.n.FSNamesystem Stopping services started for standby state
   [junit4]   2> 83287 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 83389 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl Stopping JobHistoryServer metrics system...
   [junit4]   2> 83391 ERROR (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:236)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 83405 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 83405 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 83406 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_562707F3EC0A0F3D-001
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sq_AL, timezone=Etc/GMT0
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=8,free=120967240,total=343408640
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineReducerTest, MapReduceIndexerToolArgumentParserTest, MorphlineGoLiveMiniMRTest]
   [junit4] Completed [7/7] on J1 in 76.59s, 2 tests, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:467: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 7 suites, 35 tests, 2 errors, 1 ignored [seed: 562707F3EC0A0F3D]

Total time: 215 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1034 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1034/

1 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([6B493944E699B5AC:EC1E44CB77C0C92C]:0)
	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)




Build Log:
[...truncated 8761 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=6B493944E699B5AC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ms -Dtests.timezone=Europe/Lisbon -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   2438s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6B493944E699B5AC:EC1E44CB77C0C92C]:0)
   [junit4]    > 	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
   [junit4]    > 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/J1/temp/lucene.search.TestGeoPointQuery_6B493944E699B5AC-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {string=FST50, id=FST50, point=PostingsFormat(name=Direct)}, docValues:{id=DocValuesFormat(name=SimpleText), point=DocValuesFormat(name=Asserting)}, sim=DefaultSimilarity, locale=ms, timezone=Europe/Lisbon
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=331360376,total=398983168
   [junit4]   2> NOTE: All tests run in this JVM: [TestSlowFuzzyQuery2, TestGeoPointQuery]
   [junit4] Completed [19/20] on J1 in 2892.41s, 23 tests, 1 error <<< FAILURES!

[...truncated 4 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/temp/junit4-J1-20151202_125238_127.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid29086.hprof ...
   [junit4] Heap dump file created [555886611 bytes in 4.516 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 20 suites, 142 tests, 1 error, 1 ignored (1 assumption) [seed: 6B493944E699B5AC]

Total time: 259 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1033 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1033/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=22727, name=zkCallback-553-thread-15, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=22727, name=zkCallback-553-thread-15, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
116 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=22487, name=searcherExecutor-3572-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=19958, name=qtp129174011-19958-selector-ServerConnectorManager@9c681d/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=20335, name=searcherExecutor-1760-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=19965, name=Scheduler-1630664890, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=19981, name=searcherExecutor-1498-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=20257, name=searcherExecutor-1666-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=22474, name=searcherExecutor-3577-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=22980, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1028)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.waitForLeaderToSeeDownState(ZkController.java:1597)         at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:443)         at org.apache.solr.cloud.ZkController.access$100(ZkController.java:89)         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:272)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:158)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=22440, name=searcherExecutor-3525-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=22301, name=qtp129174011-22301, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=22229, name=searcherExecutor-3399-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=20435, name=qtp2011327979-20435-selector-ServerConnectorManager@1be9f792/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=22563, name=searcherExecutor-3606-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=22228, name=searcherExecutor-3404-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=20666, name=searcherExecutor-1988-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=20437, name=qtp2011327979-20437-selector-ServerConnectorManager@1be9f792/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=22465, name=searcherExecutor-3503-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=20469, name=Thread-17402, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1238)   19) Thread[id=20534, name=searcherExecutor-1902-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=21184, name=qtp1015471035-21184, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:970)         at org.apache.solr.cloud.ZkController.register(ZkController.java:909)         at org.apache.solr.cloud.ZkController.register(ZkController.java:852)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:696)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:750)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:617)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:221)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:180)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=20024, name=Scheduler-141426940, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=20285, name=searcherExecutor-1718-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=22438, name=searcherExecutor-3517-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=20016, name=qtp1015471035-20016-selector-ServerConnectorManager@26c9cef5/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=22386, name=searcherExecutor-3445-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=22066, name=qtp1015471035-22066, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=20645, name=searcherExecutor-1978-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=20151, name=searcherExecutor-1600-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=22481, name=searcherExecutor-3568-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=20139, name=searcherExecutor-1587-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=20442, name=Scheduler-674380372, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=22215, name=searcherExecutor-3393-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=20036, name=Thread-17187, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1238)   34) Thread[id=20350, name=searcherExecutor-1772-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=22427, name=searcherExecutor-3489-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=20543, name=searcherExecutor-1907-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=22290, name=qtp1015471035-22290, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:970)         at org.apache.solr.cloud.ZkController.register(ZkController.java:909)         at org.apache.solr.cloud.ZkController.register(ZkController.java:852)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:696)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:750)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:617)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:221)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:180)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=22394, name=searcherExecutor-3470-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=20053, name=Scheduler-969142536, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=22231, name=searcherExecutor-3406-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=22470, name=searcherExecutor-3578-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=22326, name=qtp1015471035-22326, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=22323, name=qtp129174011-22323, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=23081, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_-SendThread(127.0.0.1:50840), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)   45) Thread[id=20241, name=searcherExecutor-1668-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=20509, name=searcherExecutor-1879-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=20487, name=Scheduler-849255626, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=20671, name=searcherExecutor-1998-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=20025, name=org.eclipse.jetty.server.session.HashSessionManager@5821be24Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=22505, name=searcherExecutor-3573-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=20171, name=searcherExecutor-1636-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=20040, name=searcherExecutor-1520-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=20149, name=searcherExecutor-1597-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=20263, name=searcherExecutor-1694-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=20045, name=Scheduler-2069119407, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=22264, name=qtp129174011-22264, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=20544, name=searcherExecutor-1897-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=22496, name=searcherExecutor-3562-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=20152, name=searcherExecutor-1595-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=22391, name=searcherExecutor-3465-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=20061, name=searcherExecutor-1532-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=22430, name=searcherExecutor-3483-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=20249, name=searcherExecutor-1669-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   64) Thread[id=21237, name=qtp2011327979-21237, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1028)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:970)         at org.apache.solr.cloud.ZkController.register(ZkController.java:909)         at org.apache.solr.cloud.ZkController.register(ZkController.java:852)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:696)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:750)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:617)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:221)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:180)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   65) Thread[id=23051, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_-EventThread, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:377)         at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:250)         at org.apache.solr.cloud.ZkController.publish(ZkController.java:1228)         at org.apache.solr.cloud.ZkController.publish(ZkController.java:1140)         at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:393)         at org.apache.solr.cloud.ZkController.access$100(ZkController.java:89)         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:272)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:158)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)   66) Thread[id=20547, name=searcherExecutor-1908-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=20481, name=searcherExecutor-1866-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   68) Thread[id=20438, name=qtp2011327979-20438-acceptor-0@61111b68-ServerConnector@1d7ca08f{SSL-http/1.1}{127.0.0.1:60958}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=22569, name=searcherExecutor-3623-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   70) Thread[id=20531, name=searcherExecutor-1896-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=20258, name=searcherExecutor-1689-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   72) Thread[id=20166, name=searcherExecutor-1630-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   73) Thread[id=22371, name=searcherExecutor-3433-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   74) Thread[id=20662, name=searcherExecutor-1983-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   75) Thread[id=20506, name=searcherExecutor-1881-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   76) Thread[id=22222, name=searcherExecutor-3395-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   77) Thread[id=20499, name=searcherExecutor-1876-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   78) Thread[id=19957, name=qtp129174011-19957-selector-ServerConnectorManager@9c681d/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   79) Thread[id=23082, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   80) Thread[id=22538, name=searcherExecutor-3604-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   81) Thread[id=20363, name=searcherExecutor-1791-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   82) Thread[id=20225, name=searcherExecutor-1660-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   83) Thread[id=20533, name=searcherExecutor-1906-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   84) Thread[id=20668, name=searcherExecutor-1989-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   85) Thread[id=20399, name=searcherExecutor-1828-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   86) Thread[id=20491, name=searcherExecutor-1871-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   87) Thread[id=22385, name=searcherExecutor-3446-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   88) Thread[id=20340, name=searcherExecutor-1761-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   89) Thread[id=22320, name=qtp129174011-22320, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   90) Thread[id=22434, name=searcherExecutor-3506-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   91) Thread[id=20018, name=qtp1015471035-20018-acceptor-0@12c77bc6-ServerConnector@b324ab8{SSL-http/1.1}{127.0.0.1:51129}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   92) Thread[id=22543, name=searcherExecutor-3622-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   93) Thread[id=22541, name=searcherExecutor-3614-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   94) Thread[id=19959, name=qtp129174011-19959-acceptor-0@5b272816-ServerConnector@1c77483b{SSL-http/1.1}{127.0.0.1:45593}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   95) Thread[id=22418, name=searcherExecutor-3493-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   96) Thread[id=22469, name=searcherExecutor-3556-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   97) Thread[id=22200, name=qtp2011327979-22200, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   98) Thread[id=20505, name=searcherExecutor-1877-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   99) Thread[id=22080, name=qtp129174011-22080, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   100) Thread[id=22373, name=searcherExecutor-3434-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   101) Thread[id=19966, name=org.eclipse.jetty.server.session.HashSessionManager@

[...truncated too long message...]

lectorImpl.select(SelectorImpl.java:102)
   [junit4]    >         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
   [junit4]    >         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
   [junit4]    >         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   90) Thread[id=22427, name=searcherExecutor-3489-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   91) Thread[id=19966, name=org.eclipse.jetty.server.session.HashSessionManager@75f1a51eTimer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   92) Thread[id=20393, name=searcherExecutor-1825-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   93) Thread[id=20543, name=searcherExecutor-1907-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   94) Thread[id=20642, name=searcherExecutor-1973-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   95) Thread[id=20053, name=Scheduler-969142536, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   96) Thread[id=22394, name=searcherExecutor-3470-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B01AFA1951047BBB]:0)
   [junit4] Completed [463/552] on J1 in 3276.63s, 1 test, 3 errors <<< FAILURES!

[...truncated 261 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151201_084124_734.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid30103.hprof ...
   [junit4] Heap dump file created [611582236 bytes in 10.024 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151201_084124_734.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "zkCallback-553-thread-13"
   [junit4] <<< JVM J1: EOF ----

[...truncated 17 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 552 suites (7 ignored), 2183 tests, 2 suite-level errors, 1 error, 303 ignored (18 assumptions) [seed: B01AFA1951047BBB]

Total time: 213 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1032 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1032/

1 tests failed.
FAILED:  org.apache.solr.schema.TestCloudSchemaless.test

Error Message:
QUERY FAILED: xpath=/response/arr[@name='fields']/lst/str[@name='name'][.='newTestFieldInt447']  request=/schema/fields?wt=xml  response=<?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">172</int> </lst> <arr name="fields">   <lst>     <str name="name">_root_</str>     <str name="type">string</str>     <bool name="multiValued">false</bool>     <bool name="stored">true</bool>     <bool name="indexed">true</bool>   </lst>   <lst>     <str name="name">_version_</str>     <str name="type">long</str>     <bool name="stored">true</bool>     <bool name="indexed">true</bool>   </lst>   <lst>     <str name="name">constantField</str>     <str name="type">tdouble</str>   </lst>   <lst>     <str name="name">id</str>     <str name="type">string</str>     <bool name="required">true</bool>     <bool name="multiValued">false</bool>     <bool name="stored">true</bool>     <bool name="indexed">true</bool>     <bool name="uniqueKey">true</bool>   </lst>   <lst>     <str name="name">newTestFieldInt0</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt1</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt10</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt100</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt101</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt102</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt103</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt104</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt105</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt106</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt107</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt108</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt109</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt11</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt110</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt111</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt112</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt113</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt114</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt115</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt116</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt117</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt118</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt119</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt12</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt120</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt121</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt122</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt123</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt124</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt125</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt126</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt127</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt128</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt129</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt13</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt130</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt131</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt132</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt133</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt134</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt135</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt136</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt137</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt138</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt139</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt14</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt140</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt141</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt142</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt143</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt144</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt145</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt146</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt147</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt148</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt149</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt15</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt150</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt151</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt152</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt153</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt154</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt155</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt156</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt157</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt158</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt159</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt16</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt160</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt161</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt162</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt163</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt164</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt165</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt166</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt167</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt168</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt169</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt17</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt170</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt171</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt172</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt173</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt174</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt175</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt176</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt177</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt178</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt179</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt18</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt180</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt181</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt182</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt183</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt184</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt185</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt186</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt187</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt188</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt189</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt19</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt190</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt191</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt192</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt193</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt194</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt195</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt196</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt197</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt198</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt199</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt2</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt20</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt200</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt201</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt202</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt203</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt204</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt205</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt206</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt207</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt208</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt209</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt21</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt210</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt211</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt212</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt213</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt214</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt215</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt216</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt217</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt218</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt219</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt22</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt220</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt221</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt222</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt223</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt224</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt225</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt226</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt227</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt228</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt229</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt23</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt230</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt231</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt232</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt233</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt234</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt235</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt236</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt237</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt238</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt239</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt24</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt240</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt241</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt242</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt243</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt244</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt245</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt246</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt247</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt248</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt249</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt25</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt250</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt251</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt252</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt253</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt254</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt255</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt256</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt257</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt258</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt259</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt26</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt260</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt261</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt262</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt263</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt264</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt265</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt266</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt267</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt268</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt269</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt27</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt270</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt271</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt272</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt273</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt274</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt275</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt276</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt277</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt278</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt279</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt28</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt280</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt281</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt282</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt283</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt284</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt285</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt286</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt287</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt288</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt289</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt29</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt290</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt291</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt292</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt293</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt294</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt295</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt296</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt297</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt298</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt299</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt3</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt30</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt300</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt301</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt302</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt303</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt304</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt305</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt306</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt307</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt308</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt309</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt31</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt310</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt311</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt312</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt313</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt314</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt315</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt316</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt317</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt318</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt319</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt32</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt320</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt321</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt322</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt323</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt324</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt325</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt326</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt327</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt328</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt329</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt33</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt330</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt331</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt332</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt333</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt334</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt335</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt336</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt337</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt338</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt339</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt34</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt340</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt341</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt342</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt343</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt344</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt345</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt346</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt347</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt348</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt349</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt35</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt350</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt351</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt352</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt353</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt354</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt355</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt356</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt357</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt358</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt359</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt36</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt360</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt361</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt362</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt363</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt364</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt365</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt366</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt367</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt368</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt369</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt37</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt370</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt371</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt372</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt373</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt374</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt375</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt376</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt377</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt378</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt379</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt38</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt380</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt381</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt382</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt383</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt384</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt385</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt386</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt387</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt388</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt389</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt39</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt390</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt391</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt392</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt393</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt394</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt395</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt396</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt397</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt398</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt399</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt4</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt40</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt400</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt401</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt402</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt403</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt404</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt405</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt406</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt407</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt408</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt409</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt41</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt410</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt411</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt412</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt413</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt414</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt415</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt416</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt417</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt418</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt419</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt42</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt420</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt421</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt422</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt423</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt424</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt425</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt426</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt427</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt428</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt429</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt43</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt430</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt431</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt432</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt433</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt434</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt435</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt436</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt437</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt438</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt439</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt44</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt440</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt441</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt442</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt443</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt444</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt445</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt446</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt449</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt45</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt46</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt47</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt48</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt49</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt5</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt50</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt51</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt52</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt53</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt54</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt55</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt56</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt57</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt58</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt59</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt6</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt60</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt61</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt62</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt63</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt64</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt65</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt66</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt67</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt68</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt69</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt7</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt70</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt71</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt72</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt73</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt74</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt75</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt76</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt77</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt78</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt79</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt8</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt80</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt81</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt82</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt83</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt84</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt85</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt86</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt87</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt88</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt89</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt9</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt90</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt91</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt92</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt93</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt94</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt95</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt96</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt97</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt98</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt99</str>     <str name="type">tlong</str>   </lst> </arr> </response> 

Stack Trace:
java.lang.AssertionError: QUERY FAILED: xpath=/response/arr[@name='fields']/lst/str[@name='name'][.='newTestFieldInt447']  request=/schema/fields?wt=xml  response=<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">172</int>
</lst>
<arr name="fields">
  <lst>
    <str name="name">_root_</str>
    <str name="type">string</str>
    <bool name="multiValued">false</bool>
    <bool name="stored">true</bool>
    <bool name="indexed">true</bool>
  </lst>
  <lst>
    <str name="name">_version_</str>
    <str name="type">long</str>
    <bool name="stored">true</bool>
    <bool name="indexed">true</bool>
  </lst>
  <lst>
    <str name="name">constantField</str>
    <str name="type">tdouble</str>
  </lst>
  <lst>
    <str name="name">id</str>
    <str name="type">string</str>
    <bool name="required">true</bool>
    <bool name="multiValued">false</bool>
    <bool name="stored">true</bool>
    <bool name="indexed">true</bool>
    <bool name="uniqueKey">true</bool>
  </lst>
  <lst>
    <str name="name">newTestFieldInt0</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt1</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt10</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt100</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt101</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt102</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt103</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt104</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt105</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt106</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt107</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt108</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt109</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt11</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt110</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt111</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt112</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt113</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt114</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt115</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt116</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt117</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt118</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt119</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt12</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt120</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt121</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt122</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt123</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt124</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt125</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt126</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt127</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt128</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt129</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt13</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt130</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt131</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt132</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt133</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt134</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt135</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt136</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt137</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt138</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt139</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt14</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt140</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt141</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt142</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt143</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt144</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt145</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt146</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt147</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt148</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt149</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt15</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt150</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt151</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt152</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt153</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt154</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt155</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt156</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt157</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt158</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt159</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt16</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt160</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt161</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt162</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt163</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt164</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt165</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt166</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt167</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt168</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt169</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt17</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt170</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt171</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt172</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt173</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt174</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt175</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt176</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt177</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt178</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt179</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt18</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt180</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt181</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt182</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt183</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt184</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt185</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt186</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt187</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt188</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt189</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt19</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt190</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt191</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt192</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt193</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt194</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt195</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt196</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt197</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt198</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt199</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt2</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt20</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt200</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt201</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt202</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt203</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt204</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt205</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt206</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt207</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt208</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt209</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt21</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt210</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt211</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt212</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt213</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt214</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt215</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt216</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt217</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt218</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt219</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt22</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt220</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt221</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt222</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt223</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt224</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt225</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt226</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt227</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt228</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt229</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt23</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt230</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt231</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt232</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt233</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt234</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt235</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt236</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt237</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt238</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt239</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt24</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt240</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt241</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt242</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt243</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt244</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt245</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt246</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt247</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt248</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt249</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt25</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt250</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt251</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt252</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt253</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt254</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt255</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt256</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt257</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt258</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt259</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt26</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt260</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt261</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt262</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt263</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt264</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt265</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt266</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt267</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt268</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt269</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt27</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt270</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt271</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt272</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt273</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt274</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt275</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt276</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt277</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt278</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt279</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt28</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt280</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt281</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt282</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt283</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt284</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt285</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt286</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt287</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt288</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt289</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt29</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt290</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt291</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt292</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt293</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt294</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt295</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt296</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt297</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt298</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt299</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt3</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt30</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt300</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt301</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt302</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt303</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt304</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt305</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt306</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt307</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt308</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt309</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt31</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt310</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt311</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt312</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt313</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt314</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt315</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt316</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt317</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt318</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt319</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt32</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt320</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt321</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt322</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt323</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt324</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt325</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt326</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt327</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt328</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt329</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt33</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt330</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt331</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt332</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt333</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt334</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt335</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt336</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt337</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt338</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt339</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt34</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt340</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt341</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt342</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt343</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt344</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt345</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt346</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt347</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt348</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt349</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt35</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt350</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt351</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt352</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt353</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt354</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt355</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt356</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt357</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt358</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt359</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt36</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt360</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt361</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt362</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt363</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt364</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt365</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt366</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt367</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt368</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt369</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt37</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt370</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt371</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt372</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt373</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt374</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt375</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt376</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt377</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt378</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt379</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt38</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt380</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt381</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt382</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt383</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt384</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt385</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt386</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt387</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt388</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt389</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt39</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt390</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt391</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt392</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt393</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt394</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt395</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt396</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt397</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt398</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt399</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt4</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt40</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt400</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt401</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt402</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt403</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt404</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt405</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt406</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt407</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt408</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt409</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt41</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt410</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt411</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt412</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt413</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt414</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt415</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt416</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt417</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt418</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt419</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt42</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt420</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt421</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt422</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt423</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt424</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt425</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt426</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt427</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt428</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt429</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt43</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt430</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt431</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt432</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt433</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt434</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt435</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt436</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt437</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt438</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt439</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt44</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt440</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt441</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt442</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt443</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt444</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt445</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt446</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt449</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt45</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt46</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt47</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt48</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt49</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt5</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt50</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt51</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt52</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt53</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt54</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt55</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt56</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt57</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt58</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt59</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt6</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt60</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt61</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt62</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt63</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt64</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt65</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt66</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt67</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt68</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt69</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt7</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt70</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt71</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt72</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt73</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt74</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt75</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt76</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt77</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt78</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt79</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt8</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt80</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt81</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt82</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt83</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt84</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt85</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt86</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt87</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt88</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt89</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt9</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt90</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt91</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt92</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt93</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt94</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt95</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt96</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt97</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt98</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt99</str>
    <str name="type">tlong</str>
  </lst>
</arr>
</response>

	at __randomizedtesting.SeedInfo.seed([3F443D2AA80900AD:B71002F006F56D55]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.schema.TestCloudSchemaless.test(TestCloudSchemaless.java:149)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.1.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151130_075516_857.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151130_075516_857.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 206 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1031 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1031/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=177356, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177356, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37905: collection already exists: awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)




Build Log:
[...truncated 11392 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/init-core-data-001
   [junit4]   2> 2066769 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[EA00A48BF6BCDC00]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 2066769 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[EA00A48BF6BCDC00]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2066774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2066775 INFO  (Thread-170089) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2066775 INFO  (Thread-170089) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2066875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkTestServer start zk server on port:42201
   [junit4]   2> 2066875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2066878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2066881 INFO  (zkCallback-1839-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1cbf3146 name:ZooKeeperConnection Watcher:127.0.0.1:42201 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2066881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2066881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2066881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2066895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2066900 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2066906 INFO  (zkCallback-1840-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62e4ded3 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2066907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2066907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2066907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2066909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2066910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2066911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2066913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2066913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2066916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2066916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2066918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2066918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2066920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2066920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2066922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2066922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2066924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2066924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2066926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2066926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2066930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2066931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2066933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2066933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2066935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2066935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2066941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2066941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2066946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2066965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2066975 INFO  (zkCallback-1841-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@691a793a name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2066975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2066975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2067540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1
   [junit4]   2> 2067545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2067550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@27697030{/,null,AVAILABLE}
   [junit4]   2> 2067580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.ServerConnector Started ServerConnector@fd7facb{SSL-http/1.1}{127.0.0.1:53156}
   [junit4]   2> 2067580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server Started @2070425ms
   [junit4]   2> 2067580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=53156, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores}
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001'
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2067598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2067600 INFO  (zkCallback-1842-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@245149bc name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2067600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2067600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2067601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2067601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/solr.xml
   [junit4]   2> 2067615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores
   [junit4]   2> 2067615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer New CoreContainer 1580381518
   [junit4]   2> 2067615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001]
   [junit4]   2> 2067615 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/lib
   [junit4]   2> 2067616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2067622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2067624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2067624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2067625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42201/solr
   [junit4]   2> 2067625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2067625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2067626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2067632 INFO  (zkCallback-1844-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65e69d24 name:ZooKeeperConnection Watcher:127.0.0.1:42201 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2067632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2067632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2067641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2067660 INFO  (zkCallback-1845-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43eb8a24 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2067660 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2067662 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2067663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2067666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2067667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2067669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2067671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2067672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2067673 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2067674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2067675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53156_
   [junit4]   2> 2067675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53156_
   [junit4]   2> 2067676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2067683 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2067685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2067685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94947864418254853-127.0.0.1:53156_-n_0000000000
   [junit4]   2> 2067686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53156_
   [junit4]   2> 2067686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2067689 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer Overseer (id=94947864418254853-127.0.0.1:53156_-n_0000000000) starting
   [junit4]   2> 2067692 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2067704 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2067710 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2067711 INFO  (OverseerCollectionConfigSetProcessor-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2067711 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2067729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2067795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2067795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2067799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores
   [junit4]   2> 2067799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/, loadOnStartup=true, shard=, dataDir=data/, coreNodeName=, schema=schema.xml, configSetProperties=configsetprops.json, collection=control_collection, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1, config=solrconfig.xml, transient=false}
   [junit4]   2> 2067800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/
   [junit4]   2> 2067800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2067808 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2067808 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2067811 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2067815 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2067816 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2067816 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2067816 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2067817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2067817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2067818 INFO  (zkCallback-1845-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2067819 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2068812 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2068813 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2068813 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2068813 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2068814 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2068814 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1'
   [junit4]   2> 2068814 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2068815 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2068816 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2068822 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2068871 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2068909 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2068925 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2069374 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2069381 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2069382 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2069404 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2069407 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2069409 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2069410 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2069410 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2069410 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2069412 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2069412 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 2069414 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@783de479
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2069415 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/index
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.853515625, noCFSRatio=0.19011965434550931]
   [junit4]   2> 2069416 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6293ac8c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d254de3),segFN=segments_1,generation=1}
   [junit4]   2> 2069416 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2069420 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2069420 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2069420 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2069421 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2069424 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069426 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069427 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/
   [junit4]   2> 2069427 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2069428 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069430 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069435 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/file,/schema,standard,/admin/system,/admin/segments,/admin/properties,/update/json/docs,/admin/logging,/config,/replication,/update/csv,/get,/admin/plugins,/admin/luke,/update/json,/dump,/update,/admin/mbeans,/admin/threads,/admin/ping
   [junit4]   2> 2069437 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2069438 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2069438 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2069439 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2069439 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2069439 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@65304623
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6293ac8c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d254de3),segFN=segments_1,generation=1}
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@105d2567[collection1] main
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1519165833487581184
   [junit4]   2> 2069444 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1519165833487581184
   [junit4]   2> 2069444 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2069444 INFO  (searcherExecutor-3798-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@105d2567[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2069445 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2069449 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:53156 collection:control_collection shard:shard1
   [junit4]   2> 2069450 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2069453 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94947864418254853-core_node1-n_0000000000
   [junit4]   2> 2069454 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2069456 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2069456 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2069456 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2069457 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:53156/collection1/
   [junit4]   2> 2069457 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2069457 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:53156/collection1/ has no replicas
   [junit4]   2> 2069458 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069458 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069459 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2069464 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2069466 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:53156/collection1/ shard1
   [junit4]   2> 2069467 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 2069569 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069569 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:53156/collection1/ and leader is https://127.0.0.1:53156/collection1/
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:53156
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2069620 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 2069621 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2069738 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069738 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2069860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2069865 INFO  (zkCallback-1847-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e568b91 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2069865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2069865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2069866 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2069867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2070535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1
   [junit4]   2> 2070537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001
   [junit4]   2> 2070548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2070588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@764c07d0{/,null,AVAILABLE}
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2ec7f960{SSL-http/1.1}{127.0.0.1:36896}
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server Started @2073443ms
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostPort=36896, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/tempDir-001/jetty1}
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001'
   [junit4]   2> 2070599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2070599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2070599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2070600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070601 INFO  (zkCallback-1848-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14fd4fa2 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2070601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2070602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2070602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/solr.xml
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer New CoreContainer 941909283
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001]
   [junit4]   2> 2070621 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/lib
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2070634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2070636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42201/solr
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2070645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070645 INFO  (zkCallback-1850-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f2d958 name:ZooKeeperConnection Watcher:127.0.0.1:42201 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2070646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2070653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070654 INFO  (zkCallback-1851-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@aad92d7 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2070669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2071675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36896_
   [junit4]   2> 2071675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36896_
   [junit4]   2> 2071676 INFO  (zkCallback-1847-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071676 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071676 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071676 INFO  (zkCallback-1851-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2071680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94947864418254857-127.0.0.1:36896_-n_0000000001
   [junit4]   2> 2071680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94947864418254853-127.0.0.1:53156_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2071835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2071835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2071836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores
   [junit4]   2> 2071837 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, coreNodeName=, name=collection1, dataDir=data/, shard=, loadOnStartup=true, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1, transient=false, configSetProperties=configsetprops.json, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/}
   [junit4]   2> 2071837 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/
   [junit4]   2> 2071838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2071839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2071839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2071839 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2071839 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2071841 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36896_",
   [junit4]   2>   "base_url":"https://127.0.0.1:36896",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 2071841 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:36896_",
   [junit4]   2>   "base_url":"https://127.0.0.1:36896",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2071841 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2071841 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2071842 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2071844 INFO  (zkCallback-1847-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2071844 INFO  (zkCallback-1851-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2071844 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2071844 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1'
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2072846 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2072852 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2072866 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2072945 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2072957 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2073423 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2073437 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2073439 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2073461 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2073467 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2073471 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2073473 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2073474 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2073474 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2073476 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2073476 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1], dataDir=[null]
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@783de479
   [junit4]   2> 2073478 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/data
   [junit4]   2> 2073478 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 2073478 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/d

[...truncated too long message...]

lrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177360, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37905: collection already exists: awholynewstresscollection_collection4_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177359, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36361: collection already exists: awholynewstresscollection_collection3_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177358, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36361: Could not find collection : awholynewstresscollection_collection2_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
   [junit4]   2> 2854653 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[EA00A48BF6BCDC00]-worker) [n:127.0.0.1:36361_ c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001
   [junit4]   2> Nov 29, 2015 9:43:42 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_LY, timezone=Europe/Zurich
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=53993784,total=429916160
   [junit4]   2> NOTE: All tests run in this JVM: [ClusterStateUpdateTest, TestManagedSchemaFieldTypeResource, TestLeaderInitiatedRecoveryThread, TestAddFieldRealTimeGet, TestSolrCoreProperties, RollingRestartTest, TestSolrConfigHandlerCloud, IndexBasedSpellCheckerTest, TestDocumentBuilder, TestQueryTypes, TestInitQParser, UUIDUpdateProcessorFallbackTest, TimeZoneUtilsTest, TestFreeTextSuggestions, ChaosMonkeyNothingIsSafeTest, DataDrivenBlockJoinTest, DistributedTermsComponentTest, ZkSolrClientTest, TestExpandComponent, TestRestManager, NumericFieldsTest, BigEndianAscendingWordDeserializerTest, TestClusterStateMutator, SpellPossibilityIteratorTest, SolrIndexSplitterTest, OpenExchangeRatesOrgProviderTest, TestSystemIdResolver, TestOverriddenPrefixQueryForCustomFieldType, TestClassNameShortening, SolrPluginUtilsTest, TestSerializedLuceneMatchVersion, ShardSplitTest, SuggestComponentTest, TestStandardQParsers, TestManagedSchemaFieldResource, TestSolrQueryParserDefaultOperatorResource, HdfsBasicDistributedZkTest, SpatialRPTFieldTypeTest, DocValuesTest, TestConfigSetsAPIZkFailure, TestRuleBasedAuthorizationPlugin, BitVectorTest, TestSchemaResource, HdfsRecoverLeaseTest, TestCopyFieldCollectionResource, WordBreakSolrSpellCheckerTest, TestFiltering, FullHLLTest, TestCodecSupport, TestRequestForwarding, BadCopyFieldTest, SpellCheckCollatorTest, MigrateRouteKeyTest, TestJettySolrRunner, TestBM25SimilarityFactory, FastVectorHighlighterTest, SecurityConfHandlerTest, SpellCheckComponentTest, SpellingQueryConverterTest, MultiTermTest, SpatialFilterTest, UpdateRequestProcessorFactoryTest, SimpleMLTQParserTest, SolrInfoMBeanTest, TestExactStatsCache, DistanceFunctionTest, SimplePostToolTest, DeleteReplicaTest, TestSortingResponseWriter, TestMiniSolrCloudClusterSSL, TestManagedStopFilterFactory, TestPartialUpdateDeduplication, CollectionsAPIDistributedZkTest]
   [junit4] Completed [406/551] on J1 in 788.27s, 1 test, 1 error <<< FAILURES!

[...truncated 474 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 551 suites (7 ignored), 2177 tests, 1 error, 32 ignored (18 assumptions) [seed: EA00A48BF6BCDC00]

Total time: 211 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1030 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1030/

1 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([DA3F7411FFE8837F:526B4BCB5114EE87]:0)
	at org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:568)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:986)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11014 lines...]
   [junit4] Suite: org.apache.solr.TestDistributedSearch
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/init-core-data-001
   [junit4]   2> 2164383 INFO  (SUITE-TestDistributedSearch-seed#[DA3F7411FFE8837F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2164383 INFO  (SUITE-TestDistributedSearch-seed#[DA3F7411FFE8837F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2164666 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1
   [junit4]   2> 2164667 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@174bee47{/,null,AVAILABLE}
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@44e1e24c{HTTP/1.1}{127.0.0.1:47605}
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server Started @2167062ms
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores, hostPort=47605, hostContext=/}
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control'
   [junit4]   2> 2164670 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2164670 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2164670 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/solr.xml
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1951022864
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control]
   [junit4]   2> 2164684 WARN  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/lib
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2164685 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1, loadOnStartup=true, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/, config=solrconfig.xml, configSet=collection1, schema=schema.xml, name=collection1}
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2164688 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2164688 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2164715 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2164734 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2164777 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2164784 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2165020 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2165027 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2165029 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2165033 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165036 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165037 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165038 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165039 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1], dataDir=[null]
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19efd160
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/index/
   [junit4]   2> 2165041 WARN  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/index
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=1.17578125, floorSegmentMB=1.2587890625, forceMergeDeletesPctAllowed=25.58422740531904, segmentsPerTier=5.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.532752684939979
   [junit4]   2> 2165043 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165043 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165044 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/
   [junit4]   2> 2165044 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2165047 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165047 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2165050 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165052 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165053 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165054 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165057 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2165059 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,defaults,/get,/config,lazy,mltrh,/replication,tvrh,/debug/dump,/admin/segments,spellCheckCompRH,/update/json,/admin/properties,spellCheckCompRH_Direct,/admin/file,/admin/logging,/terms,/update/json/docs,/search-facet-def,/admin/threads,/search-facet-invariants,/admin/ping,/mlt,standard,/update,/update/csv,/admin/system,mock,spellCheckWithWordbreak,/admin/mbeans,spellCheckWithWordbreak_Direct,/admin/plugins,spellCheckCompRH1,dismax,/admin/luke
   [junit4]   2> 2165060 WARN  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2165061 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2165061 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2165061 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2165062 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=29, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@483824f8[collection1] main
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2165072 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2165099 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2165100 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2165101 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2165101 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@483824f8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2165445 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1
   [junit4]   2> 2165447 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2165448 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2cd230f9{/,null,AVAILABLE}
   [junit4]   2> 2165448 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5703ce95{HTTP/1.1}{127.0.0.1:55549}
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server Started @2167841ms
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=55549, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores}
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0'
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/solr.xml
   [junit4]   2> 2165472 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores
   [junit4]   2> 2165473 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer New CoreContainer 348463432
   [junit4]   2> 2165473 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0]
   [junit4]   2> 2165473 WARN  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/lib
   [junit4]   2> 2165473 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2165474 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2165474 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores
   [junit4]   2> 2165476 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/, name=collection1, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1, configSetProperties=configsetprops.json, configSet=collection1, config=solrconfig.xml, dataDir=data/, transient=false}
   [junit4]   2> 2165476 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/
   [junit4]   2> 2165476 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2165476 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2165477 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2165477 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2165501 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2165519 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2165557 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2165563 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2165793 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2165799 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2165801 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2165805 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165808 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165810 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165810 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1], dataDir=[null]
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19efd160
   [junit4]   2> 2165813 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data
   [junit4]   2> 2165813 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index/
   [junit4]   2> 2165813 WARN  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2165814 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 2165814 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=1.17578125, floorSegmentMB=1.2587890625, forceMergeDeletesPctAllowed=25.58422740531904, segmentsPerTier=5.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.532752684939979
   [junit4]   2> 2165816 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165816 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165816 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/
   [junit4]   2> 2165817 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2165824 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165826 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165827 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165828 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165833 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2165835 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,defaults,/get,/config,lazy,mltrh,/replication,tvrh,/debug/dump,/admin/segments,spellCheckCompRH,/update/json,/admin/properties,spellCheckCompRH_Direct,/admin/file,/admin/logging,/terms,/update/json/docs,/search-facet-def,/admin/threads,/search-facet-invariants,/admin/ping,/mlt,standard,/update,/update/csv,/admin/system,mock,spellCheckWithWordbreak,/admin/mbeans,spellCheckWithWordbreak_Direct,/admin/plugins,spellCheckCompRH1,dismax,/admin/luke
   [junit4]   2> 2165837 WARN  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2165837 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2165838 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2165838 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2165838 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=29, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2165839 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165839 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165839 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@55bb101b[collection1] main
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2165848 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2165878 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2165878 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2165880 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2165880 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2165880 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2165880 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2165881 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2165882 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@55bb101b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2165978 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2165981 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 2165981 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 0
   [junit4]   2> 2165983 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 2165983 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 0
   [junit4]   2> 2165988 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1]} 0 2
   [junit4]   2> 2165992 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1]} 0 2
   [junit4]   2> 2165995 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2]} 0 1
   [junit4]   2> 2165998 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2]} 0 1
   [junit4]   2> 2166001 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[3]} 0 0
   [junit4]   2> 2166004 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[3]} 0 0
   [junit4]   2> 2166007 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[4]} 0 1
   [junit4]   2> 2166010 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[4]} 0 0
   [junit4]   2> 2166013 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[5]} 0 0
   [junit4]   2> 2166015 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[5]} 0 0
   [junit4]   2> 2166018 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[6]} 0 0
   [junit4]   2> 2166021 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[6]} 0 0
   [junit4]   2> 2166023 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[7]} 0 0
   [junit4]   2> 2166026 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[7]} 0 0
   [junit4]   2> 2166028 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[8]} 0 0
   [junit4]   2> 2166031 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[8]} 0 0
   [junit4]   2> 2166034 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[9]} 0 0
   [junit4]   2> 2166036 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[9]} 0 0
   [junit4]   2> 2166038 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166046 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 2166047 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 2166050 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@56fc74e2[collection1] main
   [junit4]   2> 2166050 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166052 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@56fc74e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9)))}
   [junit4]   2> 2166052 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 13
   [junit4]   2> 2166054 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166062 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 2166062 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 2166065 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1b16b40d[collection1] main
   [junit4]   2> 2166065 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166066 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1b16b40d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9)))}
   [junit4]   2> 2166067 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 12
   [junit4]   2> 2166070 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[10]} 0 1
   [junit4]   2> 2166074 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[10]} 0 1
   [junit4]   2> 2166077 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[11]} 0 0
   [junit4]   2> 2166079 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[11]} 0 0
   [junit4]   2> 2166082 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[12]} 0 0
   [junit4]   2> 2166085 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[12]} 0 0
   [junit4]   2> 2166087 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[13]} 0 0
   [junit4]   2> 2166090 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[13]} 0 0
   [junit4]   2> 2166093 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1001]} 0 0
   [junit4]   2> 2166096 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1001]} 0 1
   [junit4]   2> 2166098 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[14]} 0 0
   [junit4]   2> 2166100 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[14]} 0 0
   [junit4]   2> 2166103 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[15]} 0 0
   [junit4]   2> 2166106 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[15]} 0 0
   [junit4]   2> 2166108 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[16]} 0 0
   [junit4]   2> 2166110 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[16]} 0 0
   [junit4]   2> 2166114 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[17]} 0 1
   [junit4]   2> 2166118 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[17]} 0 1
   [junit4]   2> 2166121 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[100]} 0 0
   [junit4]   2> 2166123 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[100]} 0 0
   [junit4]   2> 2166126 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[101]} 0 0
   [junit4]   2> 2166129 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[101]} 0 0
   [junit4]   2> 2166132 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[102]} 0 0
   [junit4]   2> 2166134 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[102]} 0 0
   [junit4]   2> 2166137 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[103]} 0 0
   [junit4]   2> 2166139 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[103]} 0 0
   [junit4]   2> 2166141 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[104]} 0 0
   [junit4]   2> 2166144 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[104]} 0 0
   [junit4]   2> 2166148 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[105]} 0 1
   [junit4]   2> 2166150 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[105]} 0 0
   [junit4]   2> 2166153 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[106]} 0 0
   [junit4]   2> 2166155 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[106]} 0 0
   [junit4]   2> 2166158 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[107]} 0 0
   [junit4]   2> 2166160 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[107]} 0 0
   [junit4]   2> 2166163 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[108]} 0 0
   [junit4]   2> 2166165 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[108]} 0 0
   [junit4]   2> 2166168 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[109]} 0 0
   [junit4]   2> 2166172 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[109]} 0 2
   [junit4]   2> 2166175 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[110]} 0 0
   [junit4]   2> 2166178 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[110]} 0 0
   [junit4]   2> 2166180 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[111]} 0 0
   [junit4]   2> 2166183 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[111]} 0 0
   [junit4]   2> 2166185 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[112]} 0 0
   [junit4]   2> 2166187 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[112]} 0 0
   [junit4]   2> 2166190 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[113]} 0 0
   [junit4]   2> 2166192 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[113]} 0 0
   [junit4]   2> 2166195 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[114]} 0 0
   [junit4]   2> 2166197 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[114]} 0 0
   [junit4]   2> 2166199 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[115]} 0 0
   [junit4]   2> 2166202 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[115]} 0 0
   [junit4]   2> 2166204 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[116]} 0 0
   [junit4]   2> 2166207 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[116]} 0 0
   [junit4]   2> 2166209 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[117]} 0 0
   [junit4]   2> 2166211 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[117]} 0 0
   [junit4]   2> 2166214 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[118]} 0 0
   [junit4]   2> 2166216 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[118]} 0 0
   [junit4]   2> 2166219 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[119]} 0 0
   [junit4]   2> 2166221 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[119]} 0 0
   [junit4]   2> 2166223 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[120]} 0 0
   [junit4]   2> 2166226 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[120]} 0 0
   [junit4]   2> 2166228 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[121]} 0 0
   [junit4]   2> 2166231 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[121]} 0 0
   [junit4]   2> 2166233 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[122]} 0 0
   [junit4]   2> 2166235 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[122]} 0 0
   [junit4]   2> 2166238 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[123]} 0 0
   [junit4]   2> 2166240 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[123]} 0 0
   [junit4]   2> 2166243 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[124]} 0 0
   [junit4]   2> 2166245 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[124]} 0 0
   [junit4]   2> 2166248 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[125]} 0 0
   [junit4]   2> 2166250 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[125]} 0 0
   [junit4]   2> 2166253 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[126]} 0 0
   [junit4]   2> 2166255 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[126]} 0 0
   [junit4]   2> 2166258 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[127]} 0 0
   [junit4]   2> 2166260 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[127]} 0 0
   [junit4]   2> 2166262 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[128]} 0 0
   [junit4]   2> 2166265 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[128]} 0 0
   [junit4]   2> 2166267 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[129]} 0 0
   [junit4]   2> 2166269 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[129]} 0 0
   [junit4]   2> 2166272 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[130]} 0 0
   [junit4]   2> 2166274 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[130]} 0 0
   [junit4]   2> 2166277 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[131]} 0 0
   [junit4]   2> 2166279 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[131]} 0 0
   [junit4]   2> 2166281 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[132]} 0 0
   [junit4]   2> 2166284 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[132]} 0 0
   [junit4]   2> 2166286 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[133]} 0 0
   [junit4]   2> 2166289 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[133]} 0 0
   [junit4]   2> 2166291 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[134]} 0 0
   [junit4]   2> 2166293 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[134]} 0 0
   [junit4]   2> 2166296 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[135]} 0 0
   [junit4]   2> 2166298 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[135]} 0 0
   [junit4]   2> 2166300 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[136]} 0 0
   [junit4]   2> 2166303 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[136]} 0 0
   [junit4]   2> 2166305 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[137]} 0 0
   [junit4]   2> 2166308 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[137]} 0 0
   [junit4]   2> 2166310 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[138]} 0 0
   [junit4]   2> 2166313 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[138]} 0 0
   [junit4]   2> 2166317 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[139]} 0 0
   [junit4]   2> 2166320 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[139]} 0 0
   [junit4]   2> 2166322 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[140]} 0 0
   [junit4]   2> 2166325 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[140]} 0 0
   [junit4]   2> 2166327 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[141]} 0 0
   [junit4]   2> 2166330 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[141]} 0 0
   [junit4]   2> 2166333 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[142]} 0 0
   [junit4]   2> 2166335 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[142]} 0 0
   [junit4]   2> 2166338 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[143]} 0 0
   [junit4]   2> 2166340 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[143]} 0 0
   [junit4]   2> 2166342 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[144]} 0 0
   [junit4]   2> 2166345 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[144]} 0 0
   [junit4]   2> 2166347 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[145]} 0 0
   [junit4]   2> 2166350 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[145]} 0 0
   [junit4]   2> 2166352 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[146]} 0 0
   [junit4]   2> 2166355 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[146]} 0 0
   [junit4]   2> 2166358 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[147]} 0 0
   [junit4]   2> 2166361 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[147]} 0 0
   [junit4]   2> 2166364 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[148]} 0 0
   [junit4]   2> 2166367 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[148]} 0 0
   [junit4]   2> 2166369 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[149]} 0 0
   [junit4]   2> 2166372 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[149]} 0 0
   [junit4]   2> 2166374 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166393 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_3,generation=3}
   [junit4]   2> 2166393 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 2166394 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7c1a7d46[collection1] main
   [junit4]   2> 2166394 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166396 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7c1a7d46[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9) Uninverting(_1(5.5.0):C59)))}
   [junit4]   2> 2166396 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 21
   [junit4]   2> 2166398 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166414 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_3,generation=3}
   [junit4]   2> 2166414 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 2166415 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@77abb39e[collection1] main
   [junit4]   2> 2166415 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166416 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@77abb39e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9) Uninverting(_1(5.5.0):C59)))}
   [junit4]   2> 2166417 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 18
   [junit4]   2> 2166419 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.c.SolrCore Ignoring exception matching parameter cannot be negative
   [junit4]   2> 2166419 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={start=-1&q=*&version=2&wt=javabin&distrib=false} status=400 QTime=0 
   [junit4]   2> 2166422 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.c.SolrCore Ignoring exception matching parameter cannot be negative
   [junit4]   2> 2166422 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={start=0&q=*&version=2&wt=javabin&distrib=false&rows=-1} status=400 QTime=0 
   [junit4]   2> 2166425 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&distrib=false} hits=68 status=0 QTime=1 
   [junit4]   2> 2166509 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185112&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=1 
   [junit4]   2> 2166514 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185112&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javabin&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&distrib=false} status=0 QTime=0 
   [junit4]   2> 2166518 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&shards=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/} hits=68 status=0 QTime=88 
   [junit4]   2> 2166519 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.BaseDistributedSearchTestCase starting stress...
   [junit4]   2> 2166523 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185203&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=0 
   [junit4]   2> 2166526 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185203&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javabin&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&distrib=false} status=0 QTime=0 
   [junit4]   2> 2166527 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&shards=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/} hits=68 status=0 QTime=6 
   [junit4]   2> 2166704 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185383&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=0 
   [junit4]   2> 2166707 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185383&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javabin&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&distrib=false} status=0 QTime=0 
   [junit4]   2> 2166707 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&shards=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/} hits=68 status=0 QTime=6 
   [junit4]   2> 2166776 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185203&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=0 
   [junit4]   2> 2166780 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185203&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javab

[...truncated too long message...]

s/collection1/data/index;done=false>>]
   [junit4]   2> 2268491 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 2268493 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2bf26ef8{/,null,UNAVAILABLE}
   [junit4]   2> 2268500 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@1617e798{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2268500 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1743729531
   [junit4]   2> 2268500 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4681538b
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=34,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2268502 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data/index;done=false>>]
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data/index
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data;done=false>>]
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data
   [junit4]   2> 2268517 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5307e9ac{/,null,UNAVAILABLE}
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDistributedSearch -Dtests.method=test -Dtests.seed=DA3F7411FFE8837F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_MX -Dtests.timezone=America/Maceio -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    104s J2 | TestDistributedSearch.test <<<
   [junit4]    > Throwable #1: java.lang.NullPointerException
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DA3F7411FFE8837F:526B4BCB5114EE87]:0)
   [junit4]    > 	at org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:568)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:986)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2268521 INFO  (SUITE-TestDistributedSearch-seed#[DA3F7411FFE8837F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_MX, timezone=America/Maceio
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=205244096,total=487587840
   [junit4]   2> NOTE: All tests run in this JVM: [QueryParsingTest, TestInfoStreamLogging, TestLeaderElectionZkExpiry, TestJsonRequest, TestReloadDeadlock, TestOrdValues, TestManagedSchemaDynamicFieldResource, TestSolrDeletionPolicy1, ScriptEngineTest, ShowFileRequestHandlerTest, TestReversedWildcardFilterFactory, TestZkChroot, TestSuggestSpellingConverter, UpdateParamsTest, TestRandomRequestDistribution, TestIndexSearcher, TestJmxMonitoredMap, TestMiniSolrCloudCluster, TestGroupingSearch, FieldAnalysisRequestHandlerTest, MultiThreadedOCPTest, TestComplexPhraseQParserPlugin, SolrXmlInZkTest, LeaderElectionTest, TestSolrJ, TestIntervalFaceting, DistributedFacetPivotSmallTest, DirectSolrConnectionTest, TestSchemaVersionResource, BlockCacheTest, MergeStrategyTest, ForceLeaderTest, DistributedQueryComponentCustomSortTest, HardAutoCommitTest, TestRestoreCore, SliceStateTest, AssignTest, DocExpirationUpdateProcessorFactoryTest, SparseHLLTest, RegexBoostProcessorTest, PluginInfoTest, CollectionStateFormat2Test, TestSolrIndexConfig, TestRandomCollapseQParserPlugin, PrimitiveFieldTypeTest, SuggesterTest, TestStressRecovery, TestFileDictionaryLookup, RemoteQueryErrorTest, DirectUpdateHandlerOptimizeTest, RequestHandlersTest, BlockDirectoryTest, QueryResultKeyTest, SuggesterTSTTest, BasicDistributedZk2Test, CoreAdminRequestStatusTest, TestConfigOverlay, TestConfigSetsAPI, TestTrackingShardHandlerFactory, OverseerTest, CollectionsAPIAsyncDistributedZkTest, OverseerStatusTest, TestScoreJoinQPScore, CurrencyFieldXmlFileTest, SharedFSAutoReplicaFailoverUtilsTest, AliasIntegrationTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestAuthenticationFramework, UnloadDistributedZkTest, CustomCollectionTest, HighlighterConfigTest, TestIndexingPerformance, TestReplicaProperties, TestMiniSolrCloudClusterBase, TestHashQParserPlugin, HdfsSyncSliceTest, FieldMutatingUpdateProcessorTest, TestCloudManagedSchemaConcurrent, TestBlendedInfixSuggestions, AddSchemaFieldsUpdateProcessorFactoryTest, DirectUpdateHandlerTest, TestCloudInspectUtil, TestCharFilters, AtomicUpdatesTest, TestCoreDiscovery, DateRangeFieldTest, TestNoOpRegenerator, TestExactSharedStatsCache, TestExtendedDismaxParser, ClusterStateUpdateTest, TestRandomDVFaceting, TestDistributedSearch]
   [junit4] Completed [295/551] on J2 in 104.16s, 1 test, 1 error <<< FAILURES!

[...truncated 832 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 551 suites (7 ignored), 2177 tests, 1 error, 32 ignored (18 assumptions) [seed: DA3F7411FFE8837F]

Total time: 183 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1029 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1029/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:41067/jv_rk/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:41067/jv_rk/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([AF61550CE06C52F5:27356AD64E903F0D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11454 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/init-core-data-001
   [junit4]   2> 3765093 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jv_rk/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3765166 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3765179 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3765185 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3765213 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_58696_hdfs____p0ucca/webapp
   [junit4]   2> 3765429 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3765802 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58696
   [junit4]   2> 3766024 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3766034 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3766064 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_49743_datanode____.7cuqmg/webapp
   [junit4]   2> 3766361 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3766770 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49743
   [junit4]   2> 3766951 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3766953 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3766977 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40513_datanode____.r6gmm6/webapp
   [junit4]   2> 3767038 INFO  (IPC Server handler 2 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-205df9cd-d3d4-43c6-a6f3-2feea85c39cc node DatanodeRegistration(127.0.0.1, datanodeUuid=04deb699-e0b8-4712-9514-a2f7c3f391ac, infoPort=49743, ipcPort=45445, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3767038 INFO  (IPC Server handler 2 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-ac0a84a6-bb67-44fb-a723-4ccf84b541fc node DatanodeRegistration(127.0.0.1, datanodeUuid=04deb699-e0b8-4712-9514-a2f7c3f391ac, infoPort=49743, ipcPort=45445, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3767242 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3767440 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40513
   [junit4]   2> 3767953 INFO  (IPC Server handler 9 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-0577c8ab-aa7d-4a02-8452-0f9cecba68b4 node DatanodeRegistration(127.0.0.1, datanodeUuid=71f520ad-6972-46cb-b10e-08747223bbc0, infoPort=40513, ipcPort=42525, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3767953 INFO  (IPC Server handler 9 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-19b96dcd-eb77-4930-b2a3-8e208cb3efbc node DatanodeRegistration(127.0.0.1, datanodeUuid=71f520ad-6972-46cb-b10e-08747223bbc0, infoPort=40513, ipcPort=42525, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3768186 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3768186 INFO  (Thread-57490) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3768187 INFO  (Thread-57490) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3768286 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkTestServer start zk server on port:49674
   [junit4]   2> 3768286 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768302 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768321 INFO  (zkCallback-1626-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bc336cf name:ZooKeeperConnection Watcher:127.0.0.1:49674 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768321 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768321 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768322 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3768324 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768336 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768339 INFO  (zkCallback-1627-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3bdc04a5 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768340 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768340 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768340 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3768341 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3768343 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3768345 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3768348 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3768348 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3768350 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3768350 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3768352 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3768352 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3768353 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3768353 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3768355 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3768355 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3768356 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3768357 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3768362 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3768362 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3768363 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3768363 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3768364 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3768365 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3768366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3768366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3768367 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3768367 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3768811 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1
   [junit4]   2> 3768813 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3768864 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e48bd6f{/jv_rk,null,AVAILABLE}
   [junit4]   2> 3768864 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@798cf2c4{HTTP/1.1}{127.0.0.1:50813}
   [junit4]   2> 3768864 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server Started @3771672ms
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:45072/hdfs__localhost_45072__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001_tempDir-002_control_data, hostContext=/jv_rk, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores, hostPort=50813}
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001'
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768867 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768873 INFO  (zkCallback-1628-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37b39f28 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768873 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768874 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768875 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3768875 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/solr.xml
   [junit4]   2> 3768887 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores
   [junit4]   2> 3768887 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer New CoreContainer 473949852
   [junit4]   2> 3768887 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001]
   [junit4]   2> 3768888 WARN  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/lib
   [junit4]   2> 3768888 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3768889 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3768889 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49674/solr
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768891 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768909 INFO  (zkCallback-1630-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e273684 name:ZooKeeperConnection Watcher:127.0.0.1:49674 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768910 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768910 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768937 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768957 INFO  (zkCallback-1631-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3afbeb1e name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768957 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768958 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3768960 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3768962 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3768963 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3768964 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3768966 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3768967 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3768967 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3768968 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3768969 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50813_jv_rk
   [junit4]   2> 3768969 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50813_jv_rk
   [junit4]   2> 3768970 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3768971 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3768972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3768972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000
   [junit4]   2> 3768972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50813_jv_rk
   [junit4]   2> 3768973 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3768973 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer Overseer (id=94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) starting
   [junit4]   2> 3768975 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3768979 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3768985 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3768986 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3768996 INFO  (OverseerCollectionConfigSetProcessor-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3769005 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3769034 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3769035 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3769035 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores
   [junit4]   2> 3769036 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=control_collection, dataDir=data/, transient=false, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1/, shard=, config=solrconfig.xml, coreNodeName=, name=collection1, configSetProperties=configsetprops.json, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1}
   [junit4]   2> 3769037 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1/
   [junit4]   2> 3769037 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3769048 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3769048 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3769065 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3769065 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3769066 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 3769066 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3769066 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 3769067 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3769067 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3769068 INFO  (zkCallback-1631-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3769085 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1'
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3770070 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3770076 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3770104 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 3770179 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3770186 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3770639 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3770647 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3770648 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3770662 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3770665 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3770667 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3770668 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3770668 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3770668 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3770669 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3770669 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45072/solr_hdfs_home
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@14b1666d
   [junit4]   2> 3770671 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3770685 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3770686 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3770696 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3770696 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3770704 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 3770704 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 3770707 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 3770707 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3770708 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3770837 INFO  (IPC Server handler 1 on 45072) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56716 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-ac0a84a6-bb67-44fb-a723-4ccf84b541fc:NORMAL:127.0.0.1:58382|RBW], ReplicaUnderConstruction[[DISK]DS-19b96dcd-eb77-4930-b2a3-8e208cb3efbc:NORMAL:127.0.0.1:56716|FINALIZED]]} size 0
   [junit4]   2> 3770841 INFO  (IPC Server handler 8 on 45072) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58382 is added to blk_1073741825_1001 size 71
   [junit4]   2> 3770842 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b765ba46 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3b33e37d); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3770842 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3770850 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 3770853 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770853 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3770854 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770855 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770856 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770860 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/schema,/get,standard,/replication,/update/json,/admin/system,/admin/luke,/admin/threads,/update,/admin/logging,/dump,/admin/properties,/admin/ping,/admin/file,/admin/mbeans,/admin/plugins,/admin/segments,/update/json/docs,/config
   [junit4]   2> 3770861 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3770862 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3770862 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 3770873 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3770873 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3770877 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=33.234375, floorSegmentMB=1.8076171875, forceMergeDeletesPctAllowed=17.689197199279555, segmentsPerTier=12.0, maxCFSSegmentSizeMB=1.9453125, noCFSRatio=0.0
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b765ba46 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3b33e37d); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3ab1ee4b[collection1] main
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3770895 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3770895 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3770896 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3770896 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3770896 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3770897 INFO  (searcherExecutor-5955-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3ab1ee4b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518976628010516480
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1518976628010516480
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3770899 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3770907 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:50813/jv_rk collection:control_collection shard:shard1
   [junit4]   2> 3770908 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3770911 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94936039073054724-core_node1-n_0000000000
   [junit4]   2> 3770911 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3770913 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50813/jv_rk/collection1/
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3770914 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3770914 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3770917 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:50813/jv_rk/collection1/ has no replicas
   [junit4]   2> 3770919 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3770920 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3770924 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 3770925 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50813/jv_rk/collection1/ shard1
   [junit4]   2> 3771026 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771026 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771067 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:50813/jv_rk/collection1/ and leader is http://127.0.0.1:50813/jv_rk/collection1/
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50813/jv_rk
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3771079 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 3771079 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 3771159 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771169 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7115e362 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771169 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771169 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3771170 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3771171 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3771182 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771182 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771182 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771667 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1
   [junit4]   2> 3771668 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001
   [junit4]   2> 3771681 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3771766 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d11a483{/jv_rk,null,AVAILABLE}
   [junit4]   2> 3771766 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4c86bedf{HTTP/1.1}{127.0.0.1:41067}
   [junit4]   2> 3771766 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server Started @3774574ms
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:45072/hdfs__localhost_45072__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001_tempDir-002_jetty1, hostContext=/jv_rk, hostPort=41067, solrconfig=solrconfig.xml}
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001'
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3771778 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771793 INFO  (zkCallback-1634-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58f690fb name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771793 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771794 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3771795 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3771795 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/solr.xml
   [junit4]   2> 3771814 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores
   [junit4]   2> 3771814 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer New CoreContainer 1906471651
   [junit4]   2> 3771814 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001]
   [junit4]   2> 3771814 WARN  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/lib
   [junit4]   2> 3771823 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49674/solr
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3771837 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771848 INFO  (zkCallback-1636-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a28e6cf name:ZooKeeperConnection Watcher:127.0.0.1:49674 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771850 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771850 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3771866 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771975 INFO  (zkCallback-1637-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5257f9e3 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771975 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771981 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3773002 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41067_jv_rk
   [junit4]   2> 3773002 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41067_jv_rk
   [junit4]   2> 3773003 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773003 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773003 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773004 INFO  (zkCallback-1637-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773005 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3773005 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94936039073054728-127.0.0.1:41067_jv_rk-n_0000000001
   [junit4]   2> 3773006 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000 to know if I could be the leader
   [junit4]   2> 3773014 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3773014 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3773017 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores
   [junit4]   2> 3773018 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1, collection=collection1, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1/, dataDir=data/, config=solrconfig.xml, configSetProperties=configsetprops.json, name=collection1, transient=false, schema=schema.xml, shard=}
   [junit4]   2> 3773018 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1/
   [junit4]   2> 3773018 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3773034 INFO  (coreLoadExecutor-5965-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3773034 INFO  (coreLoadExecutor-5965-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:41067_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41067/jv_rk",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"} current state version: 4
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:41067_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41067/jv_rk",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"}
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 3773037 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3773038 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3773038 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3773039 INFO  (zkCallback-1637-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - upda

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.1.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151127_062023_552.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151127_062023_552.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 211 minutes 38 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1028 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1028/

3 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":60346; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":60346; 
	at __randomizedtesting.SeedInfo.seed([A8505A62BBC97604:A602EE6CBA5F440B]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy109.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineBasicMiniMRTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest:     1) Thread[id=469, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineBasicMiniMRTest]         at java.lang.Object.wait(Native Method)         at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest: 
   1) Thread[id=469, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineBasicMiniMRTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([A8505A62BBC97604]:0)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42452; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42452; 
	at __randomizedtesting.SeedInfo.seed([A8505A62BBC97604:200465B815351BFC]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy111.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy112.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:402)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)




Build Log:
[...truncated 18080 lines...]
   [junit4] Suite: org.apache.solr.hadoop.MorphlineBasicMiniMRTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/init-core-data-001
   [junit4]   2> 6901 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 7064 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.c.C.deprecation dfs.permissions is deprecated. Instead, use dfs.permissions.enabled
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 7408 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 7447 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 7534 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.c.C.deprecation hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
   [junit4]   2> 7534 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 7535 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 7538 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 7541 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 26 06:30:46
   [junit4]   2> 7546 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 7546 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 7549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 7585 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 7585 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 7585 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 7586 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 7591 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 7851 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 7865 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 7866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 7866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 7868 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 7868 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 7869 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 7872 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 7872 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 7878 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 7878 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7879 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 7879 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 7890 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 7890 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 7890 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 7990 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImage Allocated new BlockPoolId: BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 8006 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1 has been successfully formatted.
   [junit4]   2> 8011 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name2 has been successfully formatted.
   [junit4]   2> 8237 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNStorageRetentionManager Going to retain 1 images with txid >= 0
   [junit4]   2> 8240 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode createNameNode []
   [junit4]   2> 8297 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 8403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl Scheduled snapshot period at 10 second(s).
   [junit4]   2> 8403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl NameNode metrics system started
   [junit4]   2> 8406 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode fs.defaultFS is hdfs://127.0.0.1:0
   [junit4]   2> 8479 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.DFSUtil Starting Web-server for hdfs at: http://localhost:0
   [junit4]   2> 8619 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 8634 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 8666 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 8671 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
   [junit4]   2> 8671 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 8671 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 8711 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
   [junit4]   2> 8713 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 8738 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 42072
   [junit4]   2> 8738 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 8820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_42072_hdfs____.xpjzih/webapp
   [junit4]   2> 9050 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 9923 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42072
   [junit4]   2> 9935 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 9935 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 9936 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 9937 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 9937 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 9938 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 26 06:30:48
   [junit4]   2> 9938 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 9938 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9939 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 9943 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 9946 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 9946 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 9947 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 9947 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 9953 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 9953 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9955 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 9955 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 9957 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 9957 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 9957 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 9962 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 9968 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name2/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 9976 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1/current
   [junit4]   2> 9976 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name2/current
   [junit4]   2> 9988 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImage No edit log streams selected.
   [junit4]   2> 10045 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImageFormatPBINode Loading 1 INodes.
   [junit4]   2> 10065 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImageFormatProtobuf Loaded FSImage in 0 seconds.
   [junit4]   2> 10066 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImage Loaded image for txid 0 from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1/current/fsimage_0000000000000000000
   [junit4]   2> 10074 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
   [junit4]   2> 10075 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSEditLog Starting log segment at 1
   [junit4]   2> 10156 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameCache initialized with 0 entries 0 lookups
   [junit4]   2> 10156 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Finished loading FSImage in 198 msecs
   [junit4]   2> 10617 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode RPC server is binding to localhost:0
   [junit4]   2> 10632 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 10660 INFO  (Socket Reader #1 for port 55392) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 55392
   [junit4]   2> 10723 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode Clients are to use localhost:55392 to access this namenode/service.
   [junit4]   2> 10738 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Registered FSNamesystemState MBean
   [junit4]   2> 10803 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 10803 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 10803 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem initializing replication queues
   [junit4]   2> 10817 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.StateChange STATE* Leaving safe mode after 0 secs
   [junit4]   2> 10817 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.StateChange STATE* Network topology has 0 racks and 0 datanodes
   [junit4]   2> 10817 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.StateChange STATE* UnderReplicatedBlocks has 0 blocks
   [junit4]   2> 10831 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Total number of blocks            = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of invalid blocks          = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of under-replicated blocks = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of  over-replicated blocks = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of blocks being written    = 0
   [junit4]   2> 10834 INFO  (Replication Queue Initializer) [    ] o.a.h.h.StateChange STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 17 msec
   [junit4]   2> 10922 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 10924 INFO  (IPC Server listener on 55392) [    ] o.a.h.i.Server IPC Server listener on 55392: starting
   [junit4]   2> 10934 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode NameNode RPC up at: localhost/127.0.0.1:55392
   [junit4]   2> 10934 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Starting services required for active state
   [junit4]   2> 10947 INFO  (CacheReplicationMonitor(2137030917)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Starting CacheReplicationMonitor with interval 30000 milliseconds
   [junit4]   2> 10947 INFO  (CacheReplicationMonitor(2137030917)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Rescanning after 16976147130 milliseconds
   [junit4]   2> 10953 INFO  (CacheReplicationMonitor(2137030917)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Scanned 0 directive(s) and 0 block(s) in 6 millisecond(s).
   [junit4]   2> 10959 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2
   [junit4]   2> 10998 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 11000 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 11008 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 11017 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:52183
   [junit4]   2> 11021 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 11021 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 11027 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 11028 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 11029 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 11029 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 11029 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 11036 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 11036 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 58014
   [junit4]   2> 11036 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 11056 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58014_datanode____b8giwn/webapp
   [junit4]   2> 11204 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 11517 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58014
   [junit4]   2> 11519 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 11520 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 11550 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 11569 INFO  (Socket Reader #1 for port 49479) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 49479
   [junit4]   2> 11581 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:49479
   [junit4]   2> 11594 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 11598 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 11618 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:55392 starting to offer service
   [junit4]   2> 11629 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 11630 INFO  (IPC Server listener on 49479) [    ] o.a.h.i.Server IPC Server listener on 49479: starting
   [junit4]   2> 11656 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4
   [junit4]   2> 11766 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 11766 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 11766 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 11767 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:47411
   [junit4]   2> 11767 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 11767 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 11769 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 11773 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 11773 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 36053
   [junit4]   2> 11773 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 11830 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36053_datanode____psqigi/webapp
   [junit4]   2> 11967 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 12304 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 12318 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12320 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12320 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12327 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12328 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12328 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12339 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36053
   [junit4]   2> 12340 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 12340 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 12340 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 12346 INFO  (Socket Reader #1 for port 54395) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 54395
   [junit4]   2> 12351 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:54395
   [junit4]   2> 12359 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 12359 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 12361 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:55392 starting to offer service
   [junit4]   2> 12365 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 12366 INFO  (IPC Server listener on 54395) [    ] o.a.h.i.Server IPC Server listener on 54395: starting
   [junit4]   2> 12387 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 12393 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12394 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12394 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12399 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12400 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12400 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12534 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12535 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12536 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12537 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12537 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12542 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12543 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12543 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12543 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12546 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12546 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12548 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1907285210;bpid=BP-1956537108-10.41.0.5-1448515846412;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1907285210;c=0;bpid=BP-1956537108-10.41.0.5-1448515846412;dnuuid=null
   [junit4]   2> 12552 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 33118d58-6a16-4f9a-b990-610d64bf8614
   [junit4]   2> 12596 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12596 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12597 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12597 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12597 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12600 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12601 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12601 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12601 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12604 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12605 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12607 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1907285210;bpid=BP-1956537108-10.41.0.5-1448515846412;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1907285210;c=0;bpid=BP-1956537108-10.41.0.5-1448515846412;dnuuid=null
   [junit4]   2> 12611 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID a6c625f4-dc0f-4707-bb26-9f39240219d6
   [junit4]   2> 12662 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current
   [junit4]   2> 12665 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current, StorageType: DISK
   [junit4]   2> 12665 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current
   [junit4]   2> 12665 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current, StorageType: DISK
   [junit4]   2> 12668 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current
   [junit4]   2> 12685 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current, StorageType: DISK
   [junit4]   2> 12686 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current
   [junit4]   2> 12686 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current, StorageType: DISK
   [junit4]   2> 12725 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hd

[...truncated too long message...]

.n.FSNamesystem Stopping services started for standby state
   [junit4]   2> 79431 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 79533 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl Stopping JobHistoryServer metrics system...
   [junit4]   2> 79537 ERROR (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:236)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 79551 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 79552 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 79553 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_A8505A62BBC97604-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=fr_CA, timezone=Antarctica/Casey
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=9,free=97731144,total=343408640
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineReducerTest, MorphlineGoLiveMiniMRTest]
   [junit4] Completed [7/7] on J1 in 80.43s, 2 tests, 1 error <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:467: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 7 suites, 35 tests, 1 suite-level error, 2 errors, 1 ignored [seed: A8505A62BBC97604]

Total time: 190 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1027 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1027/

1 tests failed.
FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=11073, name=Thread-6280, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11073, name=Thread-6280, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
	at __randomizedtesting.SeedInfo.seed([59DA3974711B98F8]:0)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:645)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:584)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:479)
	... 5 more




Build Log:
[...truncated 10686 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/init-core-data-001
   [junit4]   2> 1356205 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[59DA3974711B98F8]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_pxg/h
   [junit4]   2> 1356211 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1356211 INFO  (Thread-6112) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1356211 INFO  (Thread-6112) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1356311 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkTestServer start zk server on port:54391
   [junit4]   2> 1356312 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356312 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356317 INFO  (zkCallback-957-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@766f28c0 name:ZooKeeperConnection Watcher:127.0.0.1:54391 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356317 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356317 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356317 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1356320 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356320 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356321 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b119dc1 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356321 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356322 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356322 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1356323 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1356325 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1356326 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1356327 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1356327 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1356329 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1356329 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1356331 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1356331 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1356333 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1356333 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1356334 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1356334 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1356336 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1356336 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1356337 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1356337 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1356339 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1356339 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1356340 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1356341 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1356342 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1356342 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1356344 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1356344 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1356603 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1
   [junit4]   2> 1356604 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1356606 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@af97390{/_pxg/h,null,AVAILABLE}
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@48b167be{HTTP/1.1}{127.0.0.1:43455}
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server Started @1359146ms
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_pxg/h, hostPort=43455, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores}
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001'
   [junit4]   2> 1356609 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1356609 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1356609 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356610 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356610 INFO  (zkCallback-959-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@530350ee name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356611 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356611 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356612 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1356612 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/solr.xml
   [junit4]   2> 1356624 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores
   [junit4]   2> 1356624 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer New CoreContainer 293072664
   [junit4]   2> 1356624 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001]
   [junit4]   2> 1356624 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/lib
   [junit4]   2> 1356625 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54391/solr
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356627 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356628 INFO  (zkCallback-961-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@641a406e name:ZooKeeperConnection Watcher:127.0.0.1:54391 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356629 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356630 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356631 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356632 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@71722a5e name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356632 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356633 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1356635 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1356637 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1356639 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1356641 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1356642 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1356644 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1356645 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1356646 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1356647 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43455__pxg%2Fh
   [junit4]   2> 1356647 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43455__pxg%2Fh
   [junit4]   2> 1356648 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1356649 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1356650 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1356651 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000
   [junit4]   2> 1356651 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43455__pxg%2Fh
   [junit4]   2> 1356651 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1356652 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer Overseer (id=94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) starting
   [junit4]   2> 1356654 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1356659 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1356659 INFO  (OverseerCollectionConfigSetProcessor-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1356659 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1356659 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1356676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1356676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1356677 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores
   [junit4]   2> 1356678 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, collection=control_collection, config=solrconfig.xml, coreNodeName=, name=collection1, loadOnStartup=true, shard=, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/}
   [junit4]   2> 1356678 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/
   [junit4]   2> 1356678 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1356679 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1356679 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1356679 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1356679 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1356681 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1356681 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1356681 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1356682 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1356682 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1356683 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1'
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1357685 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1357693 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1357705 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1357749 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1357754 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1357946 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1357948 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1357949 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1357951 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1357967 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1357967 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1357967 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@288a2593
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1357968 WARN  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/index
   [junit4]   2> 1357969 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6863981439809271]
   [junit4]   2> 1357969 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@75648327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19deb1a7),segFN=segments_1,generation=1}
   [junit4]   2> 1357969 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1357970 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/
   [junit4]   2> 1357970 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1357974 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1357974 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1357974 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1357974 WARN  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1357977 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357978 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357979 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357980 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357983 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/luke,/admin/logging,/admin/mbeans,/admin/threads,/replication,/dump,/update/csv,/get,/admin/file,/admin/plugins,/admin/properties,/admin/system,/schema,/config,/update,/admin/segments,/update/json,/admin/ping,/update/json/docs
   [junit4]   2> 1357984 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1357984 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1357984 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1357985 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1357985 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1357985 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2ddbeec0
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@75648327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19deb1a7),segFN=segments_1,generation=1}
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@380eb10a[collection1] main
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1357992 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1357992 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1357994 INFO  (searcherExecutor-4047-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@380eb10a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518765970691194880
   [junit4]   2> 1357996 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1518765970691194880
   [junit4]   2> 1357996 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1357997 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1357997 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:43455/_pxg/h collection:control_collection shard:shard1
   [junit4]   2> 1357998 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1358000 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94922873051480068-core_node1-n_0000000000
   [junit4]   2> 1358001 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43455/_pxg/h/collection1/
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1358002 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43455/_pxg/h/collection1/ has no replicas
   [junit4]   2> 1358003 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1358005 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1358006 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43455/_pxg/h/collection1/ shard1
   [junit4]   2> 1358007 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1358109 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1358157 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:43455/_pxg/h/collection1/ and leader is http://127.0.0.1:43455/_pxg/h/collection1/
   [junit4]   2> 1358157 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43455/_pxg/h
   [junit4]   2> 1358157 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1358158 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1358158 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1358159 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 1358160 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 1358181 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1358181 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358183 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56e19432 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358183 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358184 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1358184 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1358185 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1358262 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1358262 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1358441 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1
   [junit4]   2> 1358441 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001
   [junit4]   2> 1358444 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1358446 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6cd2d6b4{/_pxg/h,null,AVAILABLE}
   [junit4]   2> 1358446 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7fe964e5{HTTP/1.1}{127.0.0.1:44618}
   [junit4]   2> 1358446 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server Started @1360984ms
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores, hostContext=/_pxg/h, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/tempDir-001/jetty1, hostPort=44618}
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001'
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1358448 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358449 INFO  (zkCallback-965-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e630cf9 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358449 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358449 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1358450 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1358450 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/solr.xml
   [junit4]   2> 1358462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores
   [junit4]   2> 1358462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer New CoreContainer 150031034
   [junit4]   2> 1358463 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001]
   [junit4]   2> 1358463 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/lib
   [junit4]   2> 1358463 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54391/solr
   [junit4]   2> 1358465 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1358465 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1358465 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358466 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ba85301 name:ZooKeeperConnection Watcher:127.0.0.1:54391 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358467 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358467 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1358469 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358470 INFO  (zkCallback-968-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@753257f3 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358470 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358475 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1359480 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44618__pxg%2Fh
   [junit4]   2> 1359480 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44618__pxg%2Fh
   [junit4]   2> 1359481 INFO  (zkCallback-968-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1359481 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1359481 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1359483 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359484 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94922873051480072-127.0.0.1:44618__pxg%2Fh-n_0000000001
   [junit4]   2> 1359484 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000 to know if I could be the leader
   [junit4]   2> 1359496 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1359496 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1359497 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores
   [junit4]   2> 1359498 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/, schema=schema.xml, collection=collection1, config=solrconfig.xml, dataDir=data/, shard=, coreNodeName=, configSetProperties=configsetprops.json, transient=false, name=collection1}
   [junit4]   2> 1359498 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/
   [junit4]   2> 1359498 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1359499 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1359499 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1359499 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1359499 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1359501 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1359501 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44618__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:44618/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1359501 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44618__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:44618/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1359501 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1359502 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
   [junit4]   2> 1359604 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1359604 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1359604 INFO  (zkCallback-968-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1360501 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1360501 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1360502 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1360502 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1'
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1360505 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1360511 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1360527 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1360553 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1360559 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1360762 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1360764 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1360765 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1360768 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@288a2593
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1360784 WARN  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6863981439809271]
   [junit4]   2> 1360785 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0

[...truncated too long message...]

.process(SolrRequest.java:167)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:479)
   [junit4]    > 	... 5 moreThrowable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11075, name=Thread-6282, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([59DA3974711B98F8]:0)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:645)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:584)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:479)
   [junit4]    > 	... 5 more
   [junit4]   2> 1741997 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[59DA3974711B98F8]-worker) [n:127.0.0.1:57558__pxg%2Fh c:collection2 s:shard3 r:core_node17 x:collection2_shard3_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_PY, timezone=America/Maceio
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=266325784,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexingPerformance, RequestHandlersTest, TestLeaderElectionZkExpiry, DateRangeFieldTest, FieldMutatingUpdateProcessorTest, TestReRankQParserPlugin, TestCollationField, TestGroupingSearch, TestPHPSerializedResponseWriter, AssignTest, TestSuggestSpellingConverter, TestSweetSpotSimilarityFactory, TestFieldTypeCollectionResource, AtomicUpdatesTest, TestRandomCollapseQParserPlugin, TestFileDictionaryLookup, TestSolrConfigHandlerConcurrent, ForceLeaderTest, LeaderInitiatedRecoveryOnShardRestartTest, ScriptEngineTest, CloneFieldUpdateProcessorFactoryTest, BadIndexSchemaTest, TestJmxMonitoredMap, CoreAdminHandlerTest, HdfsThreadLeakTest, TestChildDocTransformer, InfoHandlerTest, TestLRUStatsCache, TestExactSharedStatsCache, OverseerTaskQueueTest, RecoveryZkTest, SaslZkACLProviderTest, TestSearchPerf, CircularListTest, AliasIntegrationTest, PluginInfoTest, StatelessScriptUpdateProcessorFactoryTest, TestBulkSchemaConcurrent, FieldAnalysisRequestHandlerTest, TestPerFieldSimilarity, MergeStrategyTest, TestSolrIndexConfig, BigEndianAscendingWordSerializerTest, PrimitiveFieldTypeTest, ResponseHeaderTest, TestReplicaProperties, TestRecovery, ShowFileRequestHandlerTest, TestFastWriter, TestMiniSolrCloudClusterBase, TestTrackingShardHandlerFactory, TestStressReorder, SharedFSAutoReplicaFailoverTest, DistribCursorPagingTest, RecoveryAfterSoftCommitTest, TestCharFilters, TestSolrQueryParser, TestReversedWildcardFilterFactory, OverseerStatusTest, HdfsCollectionsAPIDistributedZkTest, AutoCommitTest, CurrencyFieldXmlFileTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DistributedQueryComponentCustomSortTest, FullSolrCloudDistribCmdsTest]
   [junit4] Completed [239/551] on J2 in 386.62s, 1 test, 1 error <<< FAILURES!

[...truncated 1051 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 551 suites (7 ignored), 2177 tests, 1 error, 32 ignored (18 assumptions) [seed: 59DA3974711B98F8]

Total time: 210 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1026 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1026/

1 tests failed.
FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
this IndexWriter is closed

Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:170)
	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)




Build Log:
[...truncated 1753 lines...]
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4]   2> نوف 24, 2015 2:21:42 ص com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #14,5,TGRP-TestDuelingCodecsAtNight]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]   2> 	at org.apache.lucene.util.fst.FST.pack(FST.java:1762)
   [junit4]   2> 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:503)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]   2> 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> نوف 24, 2015 2:21:42 ص com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #15,5,TGRP-TestDuelingCodecsAtNight]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]   2> 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]   2> 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]   2> 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]   2> 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]   2> 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDuelingCodecsAtNight -Dtests.method=testBigEquals -Dtests.seed=509A95B358E09389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_EG -Dtests.timezone=Indian/Maldives -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   1041s J0 | TestDuelingCodecsAtNight.testBigEquals <<<
   [junit4]    > Throwable #1: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:170)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]    > 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5879, name=Lucene Merge Thread #14, state=RUNNABLE, group=TGRP-TestDuelingCodecsAtNight]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.pack(FST.java:1762)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:503)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5880, name=Lucene Merge Thread #15, state=RUNNABLE, group=TGRP-TestDuelingCodecsAtNight]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]    > 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J0/temp/lucene.index.TestDuelingCodecsAtNight_509A95B358E09389-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=ar_EG, timezone=Indian/Maldives
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=352965168,total=450363392
   [junit4]   2> NOTE: All tests run in this JVM: [TestAttributeSource, TestDateFilter, TestPrefixInBooleanQuery, TestTimeLimitingCollector, TestTimSorter, Test2BPagedBytes, TestNumericRangeQuery64, TestSpanNearQuery, TestCollectionUtil, TestIndexWriterNRTIsCurrent, TestMultiPhraseQuery, TestSegmentReader, TestNumericDocValuesUpdates, Test4GBStoredFields, Test2BPositions, TestReusableStringReader, TestIndexWriterDelete, TestDocIdSetBuilder, TestDocsAndPositions, TestTieredMergePolicy, TestNoDeletionPolicy, TestSizeBoundedForceMerge, TestRoaringDocIdSet, Test2BPostingsBytes, TestManyFields, TestBagOfPositions, TestStressNRT, TestFieldReuse, TestCustomSearcherSort, TestBytesRefHash, TestMultiThreadTermVectors, TestMultiLevelSkipList, TestMixedCodecs, TestNGramPhraseQuery, TestSortedNumericSortField, TestPersistentSnapshotDeletionPolicy, TestPerSegmentDeletes, TestSimilarity2, TestForUtil, TestSumDocFreq, TestSortRandom, TestSloppyPhraseQuery2, TestSleepingLockWrapper, TestFilterSpans, TestPrefixCodedTerms, TestLucene50FieldInfoFormat, TestSimilarityProvider, TestDeletionPolicy, TestTermsEnum, TestSearcherManager, TestReaderClosed, TestNumericUtils, TestConjunctionDISI, TestQueryCachingPolicy, TestOrdinalMap, TestNot, TestSpanTermQuery, TestConjunctions, TestDemo, TestLRUQueryCache, TestRegexpRandom, TestTragicIndexWriterDeadlock, TestSubScorerFreqs, TestSimpleAttributeImpl, TestMultiFields, TestField, TestIndexWriterCommit, TestFileSwitchDirectory, TestPackedInts, TestDuelingCodecs, TestBoolean2, TestAtomicUpdate, TestNorms, TestDirectoryReaderReopen, TestSearchAfter, TestConcurrentMergeScheduler, TestHighCompressionMode, TestDocumentsWriterStallControl, TestBlockPostingsFormat3, TestLucene50SegmentInfoFormat, TestLucene50StoredFieldsFormatHighCompression, TestLucene53NormsFormat, TestDocInverterPerFieldErrorInfo, TestDuelingCodecsAtNight]
   [junit4] Completed [412/412] on J0 in 1067.17s, 3 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J0-20151123_202723_803.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid23371.hprof ...
   [junit4] Heap dump file created [423897737 bytes in 0.732 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 412 suites (1 ignored), 3337 tests, 1 error, 25 ignored (21 assumptions) [seed: 509A95B358E09389]

Total time: 64 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1025 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1025/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12979, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12979, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45109/xton/f: Could not find collection : awholynewstresscollection_collection5_2
	at __randomizedtesting.SeedInfo.seed([11B33928E1F9FE1A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)




Build Log:
[...truncated 10748 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/init-core-data-001
   [junit4]   2> 1326314 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11B33928E1F9FE1A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1326314 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11B33928E1F9FE1A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /xton/f
   [junit4]   2> 1326319 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1326320 INFO  (Thread-7980) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1326320 INFO  (Thread-7980) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1326420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkTestServer start zk server on port:41216
   [junit4]   2> 1326420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326424 INFO  (zkCallback-818-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@93f6910 name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1326428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326430 INFO  (zkCallback-819-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6de97fb name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1326432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1326434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1326436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1326438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1326438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1326440 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1326441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1326442 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1326444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1326446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1326447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1326448 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1326449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1326451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1326451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1326453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1326453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1326455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1326457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1326459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1326460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1326462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326463 INFO  (zkCallback-820-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@124f17d9 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326466 INFO  (zkCallback-821-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25fa1c8a name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1326468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326470 INFO  (zkCallback-822-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c7a0c53 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1326471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1326477 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1326477 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1326479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1326479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1326481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1326484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1326485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1326485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1326487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1326487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1326488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1326488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1326490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1326491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1326493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1326493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1326494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1326494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1326787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1
   [junit4]   2> 1326788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1326790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@394400b5{/xton/f,null,AVAILABLE}
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4004048b{SSL-http/1.1}{127.0.0.1:51454}
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server Started @1329811ms
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores, hostContext=/xton/f, hostPort=51454, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/tempDir-001/control/data}
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001'
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326803 INFO  (zkCallback-823-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74bdcb8d name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326803 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326803 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1326804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/solr.xml
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1839779241
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001]
   [junit4]   2> 1326819 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/lib
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1326822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1326824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1326824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41216/solr
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326827 INFO  (zkCallback-825-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11b1f65 name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326830 INFO  (zkCallback-826-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72e7ee name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1326834 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1326839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1326841 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1326843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1326846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1326847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1326848 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1326850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1326851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51454_xton%2Ff
   [junit4]   2> 1326851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51454_xton%2Ff
   [junit4]   2> 1326853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1326854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1326856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1326857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000
   [junit4]   2> 1326857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51454_xton%2Ff
   [junit4]   2> 1326857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1326859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer Overseer (id=94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) starting
   [junit4]   2> 1326861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1326867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1326868 INFO  (OverseerCollectionConfigSetProcessor-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1326868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1326868 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1326886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1326886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1326887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores
   [junit4]   2> 1326888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, dataDir=data/, name=collection1, collection=control_collection, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1, loadOnStartup=true, config=solrconfig.xml, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/, transient=false, shard=}
   [junit4]   2> 1326888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/
   [junit4]   2> 1326888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1326889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1326889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1326890 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1326890 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1326891 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 1326892 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1326892 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1326892 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1326893 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1326896 INFO  (zkCallback-826-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1327892 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1327892 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1327893 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1327893 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1'
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1327896 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1327902 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1327930 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1327960 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1327971 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1328266 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1328274 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1328276 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1328285 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1328304 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1328308 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1328313 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1328313 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1328313 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1328314 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1328314 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1328315 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e379080
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1328317 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/index
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1328318 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@259a5a5c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38a2614d),segFN=segments_1,generation=1}
   [junit4]   2> 1328318 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1328325 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1328328 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328329 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328331 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328332 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328332 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/
   [junit4]   2> 1328332 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1328337 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,/admin/file,/admin/mbeans,/replication,/admin/logging,/update,standard,/admin/plugins,/config,/admin/ping,/dump,/update/json/docs,/get,/admin/threads,/admin/segments,/update/csv,/admin/system,/admin/properties,/schema,/admin/luke
   [junit4]   2> 1328338 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1328346 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1328346 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1328347 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1328347 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1328347 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=87.654296875, floorSegmentMB=1.4326171875, forceMergeDeletesPctAllowed=10.480213125554982, segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20849498107086886
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@259a5a5c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38a2614d),segFN=segments_1,generation=1}
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2da091f1[collection1] main
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1328355 INFO  (searcherExecutor-3342-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2da091f1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518583154290458624
   [junit4]   2> 1328358 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518583154290458624
   [junit4]   2> 1328358 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1328358 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1328364 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:51454/xton/f collection:control_collection shard:shard1
   [junit4]   2> 1328365 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1328370 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94911447009853447-core_node1-n_0000000000
   [junit4]   2> 1328371 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:51454/xton/f/collection1/
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:51454/xton/f/collection1/ has no replicas
   [junit4]   2> 1328373 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1328374 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1328376 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1328378 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:51454/xton/f/collection1/ shard1
   [junit4]   2> 1328378 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1328394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328396 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@294ea488 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1328399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1328399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1328399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328405 INFO  (zkCallback-829-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17ac3a71 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328408 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1328408 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1328409 INFO  (zkCallback-826-thread-3-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328412 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:51454/xton/f/collection1/ and leader is https://127.0.0.1:51454/xton/f/collection1/
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:51454/xton/f
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1328436 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"} current state version: 2
   [junit4]   2> 1328437 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1328442 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1328552 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328552 INFO  (zkCallback-826-thread-3-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1
   [junit4]   2> 1328722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001
   [junit4]   2> 1328723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1328725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@87f3720{/xton/f,null,AVAILABLE}
   [junit4]   2> 1328734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7c8d7f0b{SSL-http/1.1}{127.0.0.1:53114}
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server Started @1331745ms
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/tempDir-001/jetty1, hostContext=/xton/f, hostPort=53114, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores}
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001'
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1328736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328738 INFO  (zkCallback-830-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@498aea07 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1328740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/solr.xml
   [junit4]   2> 1328755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores
   [junit4]   2> 1328755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1234197051
   [junit4]   2> 1328755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001]
   [junit4]   2> 1328755 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/lib
   [junit4]   2> 1328756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1328759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41216/solr
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328768 INFO  (zkCallback-832-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36bd3a48 name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328781 INFO  (zkCallback-833-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c2d8313 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1329791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53114_xton%2Ff
   [junit4]   2> 1329791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53114_xton%2Ff
   [junit4]   2> 1329792 INFO  (zkCallback-826-thread-3-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1329792 INFO  (zkCallback-833-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1329793 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1329794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1329795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94911447009853452-127.0.0.1:53114_xton%2Ff-n_0000000001
   [junit4]   2> 1329795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000 to know if I could be the leader
   [junit4]   2> 1329805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1329805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1329806 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores
   [junit4]   2> 1329807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, configSetProperties=configsetprops.json, name=collection1, collection=collection1, shard=, loadOnStartup=true, dataDir=data/, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1/, schema=schema.xml, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1}
   [junit4]   2> 1329807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1/
   [junit4]   2> 1329807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1329810 INFO  (coreLoadExecutor-3352-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1329810 INFO  (coreLoadExecutor-3352-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1329811 INFO  (TEST-Co

[...truncated too long message...]

455-core_node20-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewcollection_5/state.json
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_2/leader_elect/shard1/election/94911447009853467-core_node13-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_6/leader_elect/shard5/election/94911447009853452-core_node2-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_9/leader_elect/shard3/election/94911447009853458-core_node9-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_9/leader_elect/shard3/election/94911447009853452-core_node7-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_6/leader_elect/shard3/election/94911447009853455-core_node18-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_6/leader_elect/shard4/election/94911447009853467-core_node17-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_6/leader_elect/shard1/election/94911447009853447-core_node16-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_2/leader_elect/shard4/election/94911447009853452-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_6/leader_elect/shard1/election/94911447009853455-core_node13-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_2/leader_elect/shard2/election/94911447009853452-core_node6-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_5/leader_elect/shard1/election/94911447009853455-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_9/leader_elect/shard1/election/94911447009853458-core_node2-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	62	/solr/collections
   [junit4]   2> 	61	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=11B33928E1F9FE1A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=it -Dtests.timezone=Asia/Urumqi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    506s J2 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12979, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45109/xton/f: Could not find collection : awholynewstresscollection_collection5_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([11B33928E1F9FE1A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12974, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45109/xton/f: Could not find collection : awholynewstresscollection_collection0_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([11B33928E1F9FE1A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
   [junit4]   2> 1832794 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11B33928E1F9FE1A]-worker) [n:127.0.0.1:48850_xton%2Ff c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=DefaultSimilarity, locale=it, timezone=Asia/Urumqi
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=242498288,total=478150656
   [junit4]   2> NOTE: All tests run in this JVM: [TestPKIAuthenticationPlugin, AutoCommitTest, DataDrivenBlockJoinTest, TestSolr4Spatial2, SuggesterTSTTest, TestSearchPerf, CoreAdminHandlerTest, PluginInfoTest, TestCoreDiscovery, TestMiniSolrCloudCluster, TestCollationField, TestSearcherReuse, SolrCloudExampleTest, LukeRequestHandlerTest, StatelessScriptUpdateProcessorFactoryTest, DateMathParserTest, QueryParsingTest, TestUtils, TestReloadDeadlock, ClusterStateTest, TestSchemaSimilarityResource, TestMacros, TestRandomMergePolicy, TestSerializedLuceneMatchVersion, CollectionStateFormat2Test, TestManagedStopFilterFactory, CoreMergeIndexesAdminHandlerTest, ShowFileRequestHandlerTest, DistribDocExpirationUpdateProcessorTest, TestInitQParser, TestTolerantSearch, TestSolrConfigHandler, HdfsThreadLeakTest, TestStressReorder, StressHdfsTest, TestExpandComponent, TestSha256AuthenticationProvider, DeleteLastCustomShardedReplicaTest, CurrencyFieldXmlFileTest, TestExtendedDismaxParser, TestReversedWildcardFilterFactory, TestJsonFacets, XmlUpdateRequestHandlerTest, DistributedQueryComponentCustomSortTest, HdfsNNFailoverTest, TestReload, UpdateParamsTest, SuggesterTest, DeleteReplicaTest, OverseerTaskQueueTest, DistributedQueueTest, TestCustomDocTransformer, TestHdfsUpdateLog, TestJmxMonitoredMap, TestManagedResourceStorage, ZkStateReaderTest, LeaderElectionTest, FullSolrCloudDistribCmdsTest, SolrCoreCheckLockOnStartupTest, TestRawTransformer, ConnectionManagerTest, TestExactStatsCache, TestSolrDeletionPolicy1, BinaryUpdateRequestHandlerTest, FieldMutatingUpdateProcessorTest, BasicDistributedZk2Test, PrimitiveFieldTypeTest, HLLUtilTest, DistributedFacetPivotSmallAdvancedTest, TestWriterPerf, SSLMigrationTest, TestZkChroot, AnalyticsMergeStrategyTest, DirectUpdateHandlerOptimizeTest, TestDocBasedVersionConstraints, TestRecoveryHdfs, DistributedVersionInfoTest, RegexBoostProcessorTest, TestPHPSerializedResponseWriter, DistanceUnitsTest, HighlighterMaxOffsetTest, HdfsRecoveryZkTest, UnloadDistributedZkTest, ScriptEngineTest, TestSolrQueryParserResource, CloudExitableDirectoryReaderTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [204/549] on J2 in 507.22s, 1 test, 1 error <<< FAILURES!

[...truncated 1134 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 549 suites (7 ignored), 2166 tests, 1 error, 32 ignored (18 assumptions) [seed: 11B33928E1F9FE1A]

Total time: 204 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1024 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1024/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([99E29040E82FC4E0:4768FE3DD38DA613]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15326, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15326, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59542/da_gh: collection already exists: awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([99E29040E82FC4E0]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)




Build Log:
[...truncated 10995 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/init-core-data-001
   [junit4]   2> 1779632 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99E29040E82FC4E0]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1779632 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99E29040E82FC4E0]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /da_gh/
   [junit4]   2> 1779637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1779637 INFO  (Thread-9737) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1779637 INFO  (Thread-9737) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1779737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkTestServer start zk server on port:54804
   [junit4]   2> 1779737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1779738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1779740 INFO  (zkCallback-1094-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66e3a7c2 name:ZooKeeperConnection Watcher:127.0.0.1:54804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1779741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1779741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1779741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1779744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1779745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1779746 INFO  (zkCallback-1095-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25ac1c59 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1779746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1779746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1779746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1779748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1779750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1779751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1779753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1779753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1779756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1779756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1779758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1779759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1779760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1779761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1779762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1779763 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1779764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1779765 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1779766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1779767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1779768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1779768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1779770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1779770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1779772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1779772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1779774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1779775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1780121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1
   [junit4]   2> 1780122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1780124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b2f5fe8{/da_gh,null,AVAILABLE}
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4f177300{HTTP/1.1}{127.0.0.1:54698}
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server Started @1782908ms
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/da_gh, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores, hostPort=54698, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/tempDir-001/control/data}
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001'
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1780130 INFO  (zkCallback-1096-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40cb6525 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1780130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1780130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1780131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1780131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/solr.xml
   [junit4]   2> 1780145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores
   [junit4]   2> 1780145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer New CoreContainer 44070698
   [junit4]   2> 1780146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001]
   [junit4]   2> 1780146 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/lib
   [junit4]   2> 1780146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1780147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54804/solr
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1780150 INFO  (zkCallback-1098-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20214727 name:ZooKeeperConnection Watcher:127.0.0.1:54804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1780151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1780151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1780153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1780154 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33a9c8e4 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1780154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1780155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1780158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1780163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1780166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1780168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1780170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1780172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1780173 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1780174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1780175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54698_da_gh
   [junit4]   2> 1780176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54698_da_gh
   [junit4]   2> 1780177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1780178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1780179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1780180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94905938134827012-127.0.0.1:54698_da_gh-n_0000000000
   [junit4]   2> 1780180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54698_da_gh
   [junit4]   2> 1780180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1780182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer Overseer (id=94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) starting
   [junit4]   2> 1780183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1780189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1780190 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1780190 INFO  (OverseerCollectionConfigSetProcessor-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1780190 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1780208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1780208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1780209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores
   [junit4]   2> 1780210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1, config=solrconfig.xml, configSetProperties=configsetprops.json, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/, dataDir=data/, schema=schema.xml, shard=, name=collection1, collection=control_collection}
   [junit4]   2> 1780210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/
   [junit4]   2> 1780211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1780212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1780212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1780212 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1780212 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1780214 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1780215 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1780215 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1780215 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1780216 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1780217 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1781215 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1781215 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1781216 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1781216 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1'
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1781219 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1781225 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1781238 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1781285 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1781292 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1781529 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1781536 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1781538 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1781547 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1781550 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1781552 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1781553 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1781553 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1781553 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1781554 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1781554 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e15c1ec
   [junit4]   2> 1781556 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data
   [junit4]   2> 1781556 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1781556 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/index
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=8, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.43692037758476887]
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12484a6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@480aee0),segFN=segments_1,generation=1}
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1781558 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/
   [junit4]   2> 1781558 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1781563 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1781566 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781567 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781568 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781569 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781573 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/admin/threads,/replication,/admin/segments,/admin/plugins,/admin/ping,/dump,/config,/admin/properties,/admin/mbeans,/schema,/update/json,standard,/admin/file,/get,/update/csv,/admin/luke,/admin/system,/update/json/docs,/update
   [junit4]   2> 1781574 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1781575 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1781575 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1781576 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1781576 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1781576 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=63.697265625, floorSegmentMB=2.177734375, forceMergeDeletesPctAllowed=18.46726359604167, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12484a6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@480aee0),segFN=segments_1,generation=1}
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@16f3fe24[collection1] main
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1781580 INFO  (searcherExecutor-3519-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@16f3fe24[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518495012192518144
   [junit4]   2> 1781582 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518495012192518144
   [junit4]   2> 1781582 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1781583 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1781583 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:54698/da_gh collection:control_collection shard:shard1
   [junit4]   2> 1781584 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1781588 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94905938134827012-core_node1-n_0000000000
   [junit4]   2> 1781592 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54698/da_gh/collection1/
   [junit4]   2> 1781594 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:54698/da_gh/collection1/ has no replicas
   [junit4]   2> 1781596 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1781598 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1781600 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54698/da_gh/collection1/ shard1
   [junit4]   2> 1781601 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1781703 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1781713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1781716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1781717 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@52e4cb3f name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1781717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1781717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1781717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1781720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1781720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1781720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1781727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1781727 INFO  (zkCallback-1102-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37899c4a name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1781728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1781728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1781732 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 1781732 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1781751 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:54698/da_gh/collection1/ and leader is http://127.0.0.1:54698/da_gh/collection1/
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54698/da_gh
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1781755 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 1781756 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1781756 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1781860 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1781860 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1782032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1
   [junit4]   2> 1782033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001
   [junit4]   2> 1782034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@52b5ad1a{/da_gh,null,AVAILABLE}
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@64dcfcf0{HTTP/1.1}{127.0.0.1:39166}
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server Started @1784817ms
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=39166, hostContext=/da_gh, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001'
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1782038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1782039 INFO  (zkCallback-1103-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10354b81 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1782039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1782039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1782041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1782041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/solr.xml
   [junit4]   2> 1782055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores
   [junit4]   2> 1782055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer New CoreContainer 29968421
   [junit4]   2> 1782055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001]
   [junit4]   2> 1782055 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/lib
   [junit4]   2> 1782056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54804/solr
   [junit4]   2> 1782058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1782058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1782058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1782060 INFO  (zkCallback-1105-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32e62e29 name:ZooKeeperConnection Watcher:127.0.0.1:54804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1782060 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1782061 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1782062 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1782064 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7253d80a name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1782064 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1782069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1783075 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39166_da_gh
   [junit4]   2> 1783075 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39166_da_gh
   [junit4]   2> 1783076 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1783076 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1783076 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1783078 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1783079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94905938134827017-127.0.0.1:39166_da_gh-n_0000000001
   [junit4]   2> 1783080 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94905938134827012-127.0.0.1:54698_da_gh-n_0000000000 to know if I could be the leader
   [junit4]   2> 1783094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1783094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1783095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores
   [junit4]   2> 1783097 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1/, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, coreNodeName=, loadOnStartup=true, collection=collection1, transient=false, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1, name=collection1, schema=schema.xml}
   [junit4]   2> 1783097 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1/
   [junit4]   2> 1783098 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1783100 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1783100 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1783100 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1783100 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1783102 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1783102 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1783102 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39166_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39166/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1783103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1783103 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:39166_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39166/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1783103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1783103 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1783103 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1783206 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1783206 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1783207 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1784103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1784103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1784104 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1784104 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1'
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1784108 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1784118 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1784147 IN

[...truncated too long message...]

ctionContextBase.cancelElection(ElectionContext.java:167)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:269)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:441)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3210619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@4e4ac73a{HTTP/1.1}{127.0.0.1:50191}
   [junit4]   2> 3210619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1867ea5a{/,null,UNAVAILABLE}
   [junit4]   2> 3210621 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[99E29040E82FC4E0]) [n:127.0.0.1:50191_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33316 33316
   [junit4]   2> 3210920 INFO  (Thread-167371) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33316 33316
   [junit4]   2> 3210921 WARN  (Thread-167371) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/aliases.json
   [junit4]   2> 	10	/solr/security.json
   [junit4]   2> 	10	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	19	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	11	/solr/clusterstate.json
   [junit4]   2> 	9	/solr/collections/collection1/state.json
   [junit4]   2> 	5	/solr/overseer_elect/election/94906005542666244-127.0.0.1:52876_-n_0000000000
   [junit4]   2> 	2	/solr/collections/all_in_lir/leader_elect/shard1/election/94906005542666252-core_node2-n_0000000000
   [junit4]   2> 	2	/solr/collections/all_in_lir/leader_elect/shard1/election/94906005542666267-core_node3-n_0000000051
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	11	/solr/collections
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=99E29040E82FC4E0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=Africa/Dakar -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  403s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([99E29040E82FC4E0:4768FE3DD38DA613]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:109)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3210925 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[99E29040E82FC4E0]-worker) [n:127.0.0.1:50191_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_99E29040E82FC4E0-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=hu, timezone=Africa/Dakar
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=262252936,total=493355008
   [junit4]   2> NOTE: All tests run in this JVM: [TestCoreDiscovery, SaslZkACLProviderTest, ClusterStateTest, TestManagedSchemaFieldTypeResource, TestAuthenticationFramework, TestBlobHandler, AsyncCallRequestStatusResponseTest, SampleTest, TestLeaderInitiatedRecoveryThread, BufferStoreTest, ForceLeaderTest, TestSweetSpotSimilarityFactory, TestFileDictionaryLookup, DocValuesMultiTest, AtomicUpdatesTest, TestFastWriter, SuggesterTSTTest, TestBlendedInfixSuggestions, DebugComponentTest, AddBlockUpdateTest, HdfsBasicDistributedZkTest, CircularListTest, TestRuleBasedAuthorizationPlugin, AnalyticsMergeStrategyTest, CloneFieldUpdateProcessorFactoryTest, FacetPivotSmallTest, HardAutoCommitTest, TestElisionMultitermQuery, HdfsDirectoryFactoryTest, TestWriterPerf, TestSimpleQParserPlugin, TestSolrQueryParserResource, TestSort, NoCacheHeaderTest, PrimitiveFieldTypeTest, TestStressReorder, TestLRUStatsCache, SolrCoreCheckLockOnStartupTest, RequestLoggingTest, TestManagedResourceStorage, OverseerTaskQueueTest, TestPerFieldSimilarity, TestStandardQParsers, QueryParsingTest, TestSchemaVersionResource, TestSolrDeletionPolicy1, BigEndianAscendingWordSerializerTest, TestExtendedDismaxParser, AutoCommitTest, TestJsonFacets, TestIndexingPerformance, TestSolrIndexConfig, RankQueryTest, TestXIncludeConfig, TestRestManager, TestJmxMonitoredMap, EnumFieldTest, TestJettySolrRunner, StatsComponentTest, TestConfigOverlay, TestDynamicFieldResource, TestCustomDocTransformer, TestFieldTypeResource, TestRecovery, FullSolrCloudDistribCmdsTest, TestReplicationHandlerBackup, ShardSplitTest, TestReloadDeadlock, BinaryUpdateRequestHandlerTest, TestHashQParserPlugin, TestIndexSearcher, UpdateParamsTest, DocValuesMissingTest, TestSolrCloudWithKerberosAlt, SecurityConfHandlerTest, TestNamedUpdateProcessors, RegexBoostProcessorTest, DistributedFacetPivotLongTailTest, TestEmbeddedSolrServerConstructors, RecoveryAfterSoftCommitTest, TestDownShardTolerantSearch, BasicAuthIntegrationTest, DocExpirationUpdateProcessorFactoryTest, TestCSVResponseWriter, TestSolrQueryParser, PluginInfoTest, ShowFileRequestHandlerTest, TestSuggestSpellingConverter, ShardRoutingTest, RecoveryZkTest, LeaderElectionTest, TestZkChroot, TestDistributedSearch, SpellingQueryConverterTest, TestSolrJ, TestUtils, SliceStateTest, UUIDFieldTest, DistributedMLTComponentTest, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, CollectionStateFormat2Test, CollectionTooManyReplicasTest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest, DistribCursorPagingTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [435/548] on J2 in 402.80s, 1 test, 1 failure <<< FAILURES!

[...truncated 374 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 548 suites (7 ignored), 2165 tests, 1 error, 1 failure, 32 ignored (18 assumptions) [seed: 99E29040E82FC4E0]

Total time: 199 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1023 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1023/

13 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:55429/_u/a/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:55429/_u/a/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E:A9E8D7C6B371A586]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82049, name=OverseerThreadFactory-2368-thread-1, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82049, name=OverseerThreadFactory-2368-thread-1, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82082, name=OverseerThreadFactory-2368-thread-2, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82082, name=OverseerThreadFactory-2368-thread-2, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponse(OverseerCollectionMessageHandler.java:2395)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponse(OverseerCollectionMessageHandler.java:2389)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:2282)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.createCollection(OverseerCollectionMessageHandler.java:2104)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:221)
	at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:434)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82048, name=Scheduler-311063407, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82048, name=Scheduler-311063407, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82217, name=OverseerThreadFactory-2368-thread-5, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82217, name=OverseerThreadFactory-2368-thread-5, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82476, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82476, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53143: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82474, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82474, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82473, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82473, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82475, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82475, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53143: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=231725, name=testExecutor-8483-thread-8, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=231725, name=testExecutor-8483-thread-8, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:32849/l: Error CREATEing SolrCore 'multiunload7': Unable to create core [multiunload7] Caused by: Direct buffer memory
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:550)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10766 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/init-core-data-001
   [junit4]   2> 1058571 INFO  (SUITE-BasicDistributedZkTest-seed#[21BCE81C1D8DC87E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_u/a
   [junit4]   2> 1058575 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1058587 INFO  (Thread-7732) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1058587 INFO  (Thread-7732) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1058676 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkTestServer start zk server on port:35702
   [junit4]   2> 1058676 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1058677 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1058680 INFO  (zkCallback-951-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@677b2773 name:ZooKeeperConnection Watcher:127.0.0.1:35702 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1058680 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1058681 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1058681 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1058685 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1058685 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1058687 INFO  (zkCallback-952-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f890479 name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1058687 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1058687 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1058687 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1058690 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1058693 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1058694 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1058697 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1058697 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1058701 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1058702 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1058707 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1058707 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1058709 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1058709 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1058712 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1058712 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1058716 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1058716 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1058717 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1058718 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1058720 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1058720 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1058723 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1058723 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1058725 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1058725 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1058727 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1058727 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1059017 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1
   [junit4]   2> 1059018 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1059020 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26fb18ab{/_u/a,null,AVAILABLE}
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2f10968b{HTTP/1.1}{127.0.0.1:58806}
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.Server Started @1061961ms
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_u/a, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores, hostPort=58806, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/tempDir-001/control/data}
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001'
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1059023 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1059023 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1059024 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1059027 INFO  (zkCallback-953-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1b7cd4fc name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1059027 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1059028 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1059029 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1059029 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/solr.xml
   [junit4]   2> 1059056 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores
   [junit4]   2> 1059057 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoreContainer New CoreContainer 313840481
   [junit4]   2> 1059057 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001]
   [junit4]   2> 1059057 WARN  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/lib
   [junit4]   2> 1059057 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1059059 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1059059 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1059059 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1059060 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35702/solr
   [junit4]   2> 1059060 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1059060 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1059064 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1059067 INFO  (zkCallback-955-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b261b3a name:ZooKeeperConnection Watcher:127.0.0.1:35702 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1059067 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1059067 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1059070 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1059071 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ff98886 name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1059071 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1059072 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1059074 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1059078 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1059082 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1059085 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1059088 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1059092 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1059094 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1059097 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1059099 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58806__u%2Fa
   [junit4]   2> 1059100 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58806__u%2Fa
   [junit4]   2> 1059103 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1059105 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1059106 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1059107 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000
   [junit4]   2> 1059107 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58806__u%2Fa
   [junit4]   2> 1059107 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1059110 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer Overseer (id=94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) starting
   [junit4]   2> 1059112 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1059118 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1059118 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1059119 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1059119 INFO  (OverseerCollectionConfigSetProcessor-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1059163 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1059163 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1059169 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores
   [junit4]   2> 1059171 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1, collection=control_collection, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/, schema=schema.xml, loadOnStartup=true, name=collection1, transient=false, configSetProperties=configsetprops.json}
   [junit4]   2> 1059173 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/
   [junit4]   2> 1059175 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1059182 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1059182 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1059185 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1059187 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1059187 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1059187 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1059188 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1059189 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1059189 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1059192 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060186 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1060186 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1060187 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1060187 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1'
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1060195 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1060203 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1060218 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1060256 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1060263 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1060526 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1060535 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1060537 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1060549 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1060553 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1060557 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1060558 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1060558 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1060558 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1060560 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1060560 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1060561 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1060561 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1060562 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1060562 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1060562 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346bc66
   [junit4]   2> 1060563 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data
   [junit4]   2> 1060563 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1060563 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1060563 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/index
   [junit4]   2> 1060564 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=0.919921875, floorSegmentMB=0.5556640625, forceMergeDeletesPctAllowed=23.150084970163487, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1060564 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5ac61053 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@159e7b4),segFN=segments_1,generation=1}
   [junit4]   2> 1060564 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1060564 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/
   [junit4]   2> 1060565 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1060573 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1060573 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1060573 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1060576 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060577 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060578 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060580 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060585 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/luke,/admin/ping,/get,/update,/admin/plugins,/schema,/admin/logging,/admin/properties,/admin/file,/update/json,/replication,/update/csv,/update/json/docs,/config,/admin/system,/dump,standard,/admin/threads,/admin/segments
   [junit4]   2> 1060586 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1060586 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1060586 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1060588 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1060588 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1060588 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=6, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.3857421875, noCFSRatio=0.16249597702253232]
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5ac61053 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@159e7b4),segFN=segments_1,generation=1}
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3a817a0e[collection1] main
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1060592 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518419750852493312
   [junit4]   2> 1060594 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518419750852493312
   [junit4]   2> 1060594 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1060594 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1060594 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:58806/_u/a collection:control_collection shard:shard1
   [junit4]   2> 1060595 INFO  (searcherExecutor-2967-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3a817a0e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1060595 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1060600 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94901234297077764-core_node1-n_0000000000
   [junit4]   2> 1060601 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1060603 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58806/_u/a/collection1/
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58806/_u/a/collection1/ has no replicas
   [junit4]   2> 1060607 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1060611 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1060613 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58806/_u/a/collection1/ shard1
   [junit4]   2> 1060614 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1060693 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1060695 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1060698 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@16260ac3 name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1060698 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1060698 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1060698 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1060702 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1060717 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060717 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:58806/_u/a/collection1/ and leader is http://127.0.0.1:58806/_u/a/collection1/
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58806/_u/a
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1060767 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1060767 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1060870 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060870 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/cluste

[...truncated too long message...]

rg.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 11 moreThrowable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82475, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 11 moreThrowable #10: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53143: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4] Completed [506/548] on J1 in 7229.31s, 1 test, 11 errors <<< FAILURES!

[...truncated 129 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151121_033414_270.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid11232.hprof ...
   [junit4] Heap dump file created [642691216 bytes in 15.265 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151121_033414_270.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "OverseerThreadFactory-2368-thread-1"
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 548 suites (7 ignored), 2165 tests, 10 suite-level errors, 2 errors, 1 failure, 159 ignored (18 assumptions) [seed: 21BCE81C1D8DC87E]

Total time: 278 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1022 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1022/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([A3790642C165612C:7DF3683FFAC703DF]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}]> but was:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}, {indexVersion=1447979710547,generation=3,filelist=[_11yg.cfe, _11yg.cfs, _11yg.si, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127g.cfe, _127g.cfs, _127g.si, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}]> but was:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}, {indexVersion=1447979710547,generation=3,filelist=[_11yg.cfe, _11yg.cfs, _11yg.si, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127g.cfe, _127g.cfs, _127g.si, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([A3790642C165612C:86AE1D72B12D6F2F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1136)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10653 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/init-core-data-001
   [junit4]   2> 431197 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[A3790642C165612C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_j/q
   [junit4]   2> 431201 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 431202 INFO  (Thread-4884) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 431202 INFO  (Thread-4884) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 431302 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkTestServer start zk server on port:60407
   [junit4]   2> 431302 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431303 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431306 INFO  (zkCallback-283-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f2b0e86 name:ZooKeeperConnection Watcher:127.0.0.1:60407 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 431309 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431310 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431317 INFO  (zkCallback-284-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34095bc name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431317 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 431320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 431322 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 431324 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 431326 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 431327 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 431335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 431335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 431337 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 431338 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 431340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 431340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 431342 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 431342 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 431344 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 431344 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 431346 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 431346 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 431348 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 431349 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 431356 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 431356 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 431360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 431361 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 431365 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 431365 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 431679 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1
   [junit4]   2> 431680 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 431682 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3be6eb6f{/_j/q,null,AVAILABLE}
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7c65e893{HTTP/1.1}{127.0.0.1:59007}
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.Server Started @435173ms
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=59007, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/tempDir-001/control/data, hostContext=/_j/q, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores}
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/'
   [junit4]   2> 431684 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 431684 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 431706 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431707 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431710 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@611d491d name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431710 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431711 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431713 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 431713 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/solr.xml
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoreContainer New CoreContainer 844721785
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/]
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/lib
   [junit4]   2> 431736 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/lib).
   [junit4]   2> 431754 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 431756 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 431757 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60407/solr
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431759 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431774 INFO  (zkCallback-287-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4670da75 name:ZooKeeperConnection Watcher:127.0.0.1:60407 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431775 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431776 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431778 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431779 INFO  (zkCallback-288-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77166a8b name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431779 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431786 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 431788 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 431795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 431797 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 431800 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 431804 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 431806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 431808 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 431810 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 431811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59007__j%2Fq
   [junit4]   2> 431811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59007__j%2Fq
   [junit4]   2> 431814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 431816 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 431818 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 431819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000
   [junit4]   2> 431819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59007__j%2Fq
   [junit4]   2> 431819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 431821 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer Overseer (id=94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) starting
   [junit4]   2> 431824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 431835 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 431835 INFO  (OverseerCollectionConfigSetProcessor-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 431836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 431838 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 431862 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 431862 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 431863 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores
   [junit4]   2> 431864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1, name=collection1, schema=schema.xml, configSetProperties=configsetprops.json, coreNodeName=, loadOnStartup=true, shard=, collection=control_collection}
   [junit4]   2> 431864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/
   [junit4]   2> 431865 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 431867 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 431867 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 431875 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 431878 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 431878 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:59007/_j/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59007__j%2Fq",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 431878 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 431879 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:59007/_j/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59007__j%2Fq",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 431879 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 431880 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 431883 INFO  (zkCallback-288-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 432876 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 432876 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 432877 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 432877 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 432877 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 432878 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/'
   [junit4]   2> 432878 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 432878 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 432895 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 432902 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 432915 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 432944 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 432945 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 432952 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 433201 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 433209 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 433210 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 433222 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 433226 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 433229 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433230 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 433230 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 433230 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433231 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 433231 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@9eb620c
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 433233 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/index
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=4, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 433234 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e1d1ca4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39438b76),segFN=segments_1,generation=1}
   [junit4]   2> 433234 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 433235 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/
   [junit4]   2> 433235 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 433239 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 433239 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 433240 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 433243 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433244 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433246 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433247 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433251 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/admin/segments,/admin/mbeans,/admin/luke,/admin/system,/schema,/update,/admin/ping,/update/json,/config,/update/json/docs,standard,/get,/admin/properties,/admin/threads,/admin/plugins,/admin/file,/update/csv,/dump,/admin/logging
   [junit4]   2> 433252 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 433254 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=0.271484375, floorSegmentMB=0.232421875, forceMergeDeletesPctAllowed=12.024019232950184, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 433254 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e1d1ca4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39438b76),segFN=segments_1,generation=1}
   [junit4]   2> 433254 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 433255 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@46f2c8e2[collection1] main
   [junit4]   2> 433255 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 433258 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 433258 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518315773719740416
   [junit4]   2> 433258 INFO  (searcherExecutor-1131-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@46f2c8e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 433260 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518315773719740416
   [junit4]   2> 433260 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 433261 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 433261 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59007/_j/q collection:control_collection shard:shard1
   [junit4]   2> 433262 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 433266 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94894735722938372-core_node1-n_0000000000
   [junit4]   2> 433267 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 433269 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 433269 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59007/_j/q/collection1/
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59007/_j/q/collection1/ has no replicas
   [junit4]   2> 433271 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 433274 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 433276 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59007/_j/q/collection1/ shard1
   [junit4]   2> 433277 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59007/_j/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 433380 INFO  (zkCallback-288-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 433380 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 433381 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 433382 INFO  (zkCallback-290-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6077f0ee name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 433382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 433382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 433382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 433385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 433385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 433385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using defa

[...truncated too long message...]

/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard2_replica1/data
   [junit4]   2> 17503051 INFO  (qtp62381361-15321) [n:127.0.0.1:40675_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory Removing directory after core close: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard2_replica1/data
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=PostingsFormat(name=LuceneFixedGap), id=Lucene50(blocksize=128), _version_=PostingsFormat(name=LuceneFixedGap), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_i1=PostingsFormat(name=LuceneFixedGap), a_t=PostingsFormat(name=LuceneFixedGap), text=FSTOrd50, other_tl1=PostingsFormat(name=LuceneFixedGap), intDefault=PostingsFormat(name=LuceneFixedGap), timestamp=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=PostingsFormat(name=LuceneFixedGap), range_facet_l_dv=Lucene50(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Lucene54)}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ar_QA, timezone=AGT
   [junit4]   2> 17479997 INFO  (qtp62381361-15372) [n:127.0.0.1:40675_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard3_replica2/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard3_replica2/data;done=false>>]
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=284,free=40079432,total=495976448
   [junit4]   2> NOTE: All tests run in this JVM: [DistanceUnitsTest, HdfsSyncSliceTest, TestZkChroot, TestMissingGroups, LeaderElectionTest, TestSearchPerf, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SaslZkACLProviderTest, ClusterStateTest, TestCloudManagedSchema, TestCopyFieldCollectionResource, TestManagedSchemaDynamicFieldResource, AsyncCallRequestStatusResponseTest, TestSolrQueryParserDefaultOperatorResource, DateMathParserTest, BadComponentTest, SampleTest, TestPostingsSolrHighlighter, TestDownShardTolerantSearch, TestCollationField, TestReversedWildcardFilterFactory, RecoveryAfterSoftCommitTest, QueryResultKeyTest, DateRangeFieldTest, TestStressRecovery, TestReload, TestCoreDiscovery, DocExpirationUpdateProcessorFactoryTest, DocValuesMissingTest, BJQParserTest, LukeRequestHandlerTest, OverseerTaskQueueTest, DistributedFacetPivotSmallTest, TestConfigSetsAPIZkFailure, PreAnalyzedUpdateProcessorTest, DirectSolrConnectionTest, TestJettySolrRunner, SolrCoreCheckLockOnStartupTest, ShowFileRequestHandlerTest, LeaderInitiatedRecoveryOnShardRestartTest, TestFuzzyAnalyzedSuggestions, TestGroupingSearch, ScriptEngineTest, TestRuleBasedAuthorizationPlugin, TestSolrIndexConfig, TestSolrDeletionPolicy1, OverseerTest, TestSolrDynamicMBean, DirectUpdateHandlerTest, TestCSVResponseWriter, TestBlendedInfixSuggestions, SuggestComponentContextFilterQueryTest, CoreAdminHandlerTest, TestHashQParserPlugin, TestSolrCloudWithKerberosAlt, TestConfigSetImmutable, AutoCommitTest, BadIndexSchemaTest, CursorMarkTest, VMParamsZkACLAndCredentialsProvidersTest, TestSha256AuthenticationProvider, CircularListTest, TestSortingResponseWriter, TestIndexSearcher, CurrencyFieldXmlFileTest, TestCharFilters, TestBlobHandler, HdfsThreadLeakTest, TestElisionMultitermQuery, SuggesterTest, TestAddFieldRealTimeGet, TestAuthenticationFramework, TestOverriddenPrefixQueryForCustomFieldType, FacetPivotSmallTest, TestSweetSpotSimilarityFactory, StatsComponentTest, CollectionStateFormat2Test, PrimitiveFieldTypeTest, TestUtils, PluginInfoTest, TestReloadDeadlock, TestMiniSolrCloudCluster, UpdateParamsTest, TestConfigOverlay, DocValuesMultiTest, DeleteLastCustomShardedReplicaTest, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4]   2> 17505815 INFO  (qtp62381361-15372) [n:127.0.0.1:40675_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard3_replica2/data []
   [junit4]   2> 17479161 INFO  (qtp47016225-14852) [n:127.0.0.1:48316_f_b%2Fx c:awholynewstresscollection_collection2_0 s:shard5 r:core_node9 x:awholynewstresscollection_collection2_0_shard5_replica3] o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores params={qt=/admin/cores&action=CREATE&numShards=5&shard=shard5&version=2&collection=awholynewstresscollection_collection2_0&name=awholynewstresscollection_collection2_0_shard5_replica3&collection.configName=conf1&wt=javabin} status=0 QTime=15459256 
   [junit4]   2> 17479160 INFO  (qtp1482630108-14938) [n:127.0.0.1:52329_f_b%2Fx c:awholynewstresscollection_collection3_0 s:shard2 r:core_node18 x:awholynewstresscollection_collection3_0_shard2_replica1] o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores params={qt=/admin/cores&action=CREATE&numShards=5&shard=shard2&version=2&collection=awholynewstresscollection_collection3_0&name=awholynewstresscollection_collection3_0_shard2_replica1&collection.configName=conf1&wt=javabin} status=0 QTime=15458920 
   [junit4]   2> 17479160 INFO  (qtp1482630108-14879) [n:127.0.0.1:52329_f_b%2Fx c:awholynewstresscollection_collection2_0 s:shard1 r:core_node11 x:awholynewstresscollection_collection2_0_shard1_replica2] o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores params={qt=/admin/cores&action=CREATE&numShards=5&shard=shard1&version=2&collection=awholynewstresscollection_collection2_0&name=awholynewstresscollection_collection2_0_shard1_replica2&collection.configName=conf1&wt=javabin} status=0 QTime=15450581 
   [junit4]   2> 17479160 INFO  (qtp47016225-14927) [n:127.0.0.1:48316_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 17505816 INFO  (qtp47016225-14927) [n:127.0.0.1:48316_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/shard-1-001/cores/awholynewstresscollection_collection5_0_shard3_replica1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/shard-1-001/cores/awholynewstresscollection_collection5_0_shard3_replica1/data;done=false>>]
   [junit4]   2> 17479160 INFO  (qtp1302440199-14671) [n:127.0.0.1:32829_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4] ERROR   0.00s J2 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4] Completed [548/548] on J2 in 15569.37s, 1 test, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151120_001200_739.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18208.hprof ...
   [junit4] Heap dump file created [676079728 bytes in 48.389 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151120_001200_739.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "TEST-CollectionsAPIDistributedZkTest.test-seed#[A3790642C165612C]-SendThread(127.0.0.1:57818)"
   [junit4] <<< JVM J2: EOF ----

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 548 suites (7 ignored), 2165 tests, 1 suite-level error, 2 failures, 32 ignored (18 assumptions) [seed: A3790642C165612C]

Total time: 416 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1021 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1021/

2 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":37722; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":37722; 
	at __randomizedtesting.SeedInfo.seed([769F8DA6084321B3:78CD39A809D513BC]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy108.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy109.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":44672; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":44672; 
	at __randomizedtesting.SeedInfo.seed([769F8DA6084321B3:FECBB27CA6BF4C4B]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy111.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:402)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)




Build Log:
[...truncated 18076 lines...]
   [junit4] Suite: org.apache.solr.hadoop.MorphlineBasicMiniMRTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1153 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1324 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 1373 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 1460 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.c.C.deprecation hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
   [junit4]   2> 1460 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 1461 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 1465 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 1468 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 19 02:00:15
   [junit4]   2> 1473 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 1473 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1476 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 1476 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 1511 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 1513 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 1513 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 1513 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 1522 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 1523 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 1523 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 1523 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 1529 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 1818 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 1818 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1818 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 1819 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 1819 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 1835 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 1835 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1835 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 1836 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 1838 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 1838 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 1838 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 1841 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 1842 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 1847 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 1848 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1848 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 1848 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 1858 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 1859 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 1859 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 1963 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImage Allocated new BlockPoolId: BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 1988 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1 has been successfully formatted.
   [junit4]   2> 1993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name2 has been successfully formatted.
   [junit4]   2> 2266 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNStorageRetentionManager Going to retain 1 images with txid >= 0
   [junit4]   2> 2270 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode createNameNode []
   [junit4]   2> 2342 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2441 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl Scheduled snapshot period at 10 second(s).
   [junit4]   2> 2441 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl NameNode metrics system started
   [junit4]   2> 2445 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode fs.defaultFS is hdfs://127.0.0.1:0
   [junit4]   2> 2538 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.DFSUtil Starting Web-server for hdfs at: http://localhost:0
   [junit4]   2> 2713 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 2744 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2778 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 2783 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
   [junit4]   2> 2787 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 2787 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 2863 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
   [junit4]   2> 2866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 2894 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 36968
   [junit4]   2> 2894 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2977 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_36968_hdfs____xbxi5e/webapp
   [junit4]   2> 3207 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4270 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36968
   [junit4]   2> 4282 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 4283 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 4284 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 4284 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 4284 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 19 02:00:18
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 4305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 4305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 4305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 4306 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 4306 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4306 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 4310 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 4312 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 4313 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 4313 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 4321 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 4327 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name2/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 4332 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1/current
   [junit4]   2> 4332 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name2/current
   [junit4]   2> 4332 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImage No edit log streams selected.
   [junit4]   2> 4359 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImageFormatPBINode Loading 1 INodes.
   [junit4]   2> 4374 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImageFormatProtobuf Loaded FSImage in 0 seconds.
   [junit4]   2> 4374 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImage Loaded image for txid 0 from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1/current/fsimage_0000000000000000000
   [junit4]   2> 4382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
   [junit4]   2> 4382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSEditLog Starting log segment at 1
   [junit4]   2> 4435 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameCache initialized with 0 entries 0 lookups
   [junit4]   2> 4435 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Finished loading FSImage in 122 msecs
   [junit4]   2> 4850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode RPC server is binding to localhost:0
   [junit4]   2> 4865 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 4893 INFO  (Socket Reader #1 for port 33481) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 33481
   [junit4]   2> 4945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode Clients are to use localhost:33481 to access this namenode/service.
   [junit4]   2> 4952 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Registered FSNamesystemState MBean
   [junit4]   2> 4991 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 4991 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 4992 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem initializing replication queues
   [junit4]   2> 4992 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.StateChange STATE* Leaving safe mode after 0 secs
   [junit4]   2> 4993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.StateChange STATE* Network topology has 0 racks and 0 datanodes
   [junit4]   2> 4993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.StateChange STATE* UnderReplicatedBlocks has 0 blocks
   [junit4]   2> 4998 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Total number of blocks            = 0
   [junit4]   2> 5006 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of invalid blocks          = 0
   [junit4]   2> 5006 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of under-replicated blocks = 0
   [junit4]   2> 5007 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of  over-replicated blocks = 0
   [junit4]   2> 5007 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of blocks being written    = 0
   [junit4]   2> 5007 INFO  (Replication Queue Initializer) [    ] o.a.h.h.StateChange STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 15 msec
   [junit4]   2> 5094 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 5104 INFO  (IPC Server listener on 33481) [    ] o.a.h.i.Server IPC Server listener on 33481: starting
   [junit4]   2> 5108 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode NameNode RPC up at: localhost/127.0.0.1:33481
   [junit4]   2> 5108 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Starting services required for active state
   [junit4]   2> 5121 INFO  (CacheReplicationMonitor(1187388745)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Starting CacheReplicationMonitor with interval 30000 milliseconds
   [junit4]   2> 5121 INFO  (CacheReplicationMonitor(1187388745)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Rescanning after 16383917487 milliseconds
   [junit4]   2> 5124 INFO  (CacheReplicationMonitor(1187388745)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Scanned 0 directive(s) and 0 block(s) in 3 millisecond(s).
   [junit4]   2> 5132 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2
   [junit4]   2> 5225 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 5227 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 5236 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 5245 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:53833
   [junit4]   2> 5250 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 5250 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 5257 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 5258 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 5259 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 5259 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 5259 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 5266 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 5267 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 37520
   [junit4]   2> 5267 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 5288 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37520_datanode____.vfq9u2/webapp
   [junit4]   2> 5475 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 5953 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37520
   [junit4]   2> 5956 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 5956 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 5984 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 5984 INFO  (Socket Reader #1 for port 38809) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 38809
   [junit4]   2> 5996 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:38809
   [junit4]   2> 6010 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 6014 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 6035 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:33481 starting to offer service
   [junit4]   2> 6062 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 6067 INFO  (IPC Server listener on 38809) [    ] o.a.h.i.Server IPC Server listener on 38809: starting
   [junit4]   2> 6081 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4
   [junit4]   2> 6120 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 6120 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 6121 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 6122 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:35873
   [junit4]   2> 6122 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 6122 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 6125 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 6126 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 6127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 6127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 6127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 6129 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 6129 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 38511
   [junit4]   2> 6130 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 6166 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_38511_datanode____ynqhrr/webapp
   [junit4]   2> 6301 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 6733 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38511
   [junit4]   2> 6734 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 6734 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 6735 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 6736 INFO  (Socket Reader #1 for port 41943) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 41943
   [junit4]   2> 6740 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:41943
   [junit4]   2> 6729 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 6764 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 6764 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 6766 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6769 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6769 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 6771 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:33481 starting to offer service
   [junit4]   2> 6772 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 6772 INFO  (IPC Server listener on 41943) [    ] o.a.h.i.Server IPC Server listener on 41943: starting
   [junit4]   2> 6799 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 6800 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6802 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6802 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 6802 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6803 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6803 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 6811 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6812 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6812 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7021 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 7021 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7023 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7024 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7024 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7028 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7029 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7029 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7029 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7042 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7043 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7044 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 7045 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7046 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7046 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7046 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7050 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7051 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7051 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7051 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7055 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=2035105108;bpid=BP-1156588151-10.41.0.5-1447923616226;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=2035105108;c=0;bpid=BP-1156588151-10.41.0.5-1447923616226;dnuuid=null
   [junit4]   2> 7056 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7057 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7060 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=2035105108;bpid=BP-1156588151-10.41.0.5-1447923616226;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=2035105108;c=0;bpid=BP-1156588151-10.41.0.5-1447923616226;dnuuid=null
   [junit4]   2> 7062 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 4d668143-e162-4746-bc91-d7aa9aab5633
   [junit4]   2> 7067 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 9f1586ea-0d50-45d5-8e68-dd31c9f612be
   [junit4]   2> 7143 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current
   [junit4]   2> 7145 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current, StorageType: DISK
   [junit4]   2> 7146 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current
   [junit4]   2> 7146 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current, StorageType: DISK
   [junit4]   2> 7149 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current
   [junit4]   2> 7152 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current, StorageType: DISK
   [junit4]   2> 7156 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current
   [junit4]   2> 7159 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current, StorageType: DISK
   [junit4]   2> 7194 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Registered FSDatasetState MBean
   [junit4]   2> 7194 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Registered FSDatasetState MBean
   [junit4]   2> 7202 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/so

[...truncated too long message...]

er) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.h.s.n.FSNamesystem Stopping services started for standby state
   [junit4]   2> 79921 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 80023 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl Stopping JobHistoryServer metrics system...
   [junit4]   2> 80026 ERROR (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:236)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 80040 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 80041 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 80042 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_769F8DA6084321B3-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=no, timezone=Indian/Comoro
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=9,free=216583448,total=337641472
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineGoLiveMiniMRTest]
   [junit4] Completed [7/7] on J1 in 81.69s, 2 tests, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:467: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 7 suites, 35 tests, 2 errors, 1 ignored [seed: 769F8DA6084321B3]

Total time: 208 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1020 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1020/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Could not load collection from ZK:nodes_used_collection

Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK:nodes_used_collection
	at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1022)
	at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:555)
	at org.apache.solr.common.cloud.ClusterState.getCollections(ClusterState.java:207)
	at org.apache.solr.common.cloud.ZkStateReader.constructState(ZkStateReader.java:461)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:266)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deleteCollectionWithDownNodes(CollectionsAPIDistributedZkTest.java:251)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:163)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/nodes_used_collection/state.json
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1035)
	at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1020)
	... 51 more




Build Log:
[...truncated 11024 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/init-core-data-001
   [junit4]   2> 990323 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D89271C806E95B6A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 990323 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D89271C806E95B6A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 990328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 990329 INFO  (Thread-6443) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 990329 INFO  (Thread-6443) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 990429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkTestServer start zk server on port:39182
   [junit4]   2> 990429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990449 INFO  (zkCallback-2385-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5061d28 name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990450 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 990453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990455 INFO  (zkCallback-2386-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8a6ef51 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 990457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 990459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 990460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 990463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 990463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 990467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 990467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 990469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 990471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 990473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 990473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 990476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 990476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 990478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 990478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 990481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 990481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 990483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 990485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 990487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 990487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 990489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990492 INFO  (zkCallback-2387-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79bc1b05 name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 990494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990496 INFO  (zkCallback-2388-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14be6b6c name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 990497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 990500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 990500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 990502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 990502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 990504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 990506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 990511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 990511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 990515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 990515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 990517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 990517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 990519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 990522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 990524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 990524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 990528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 990528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 990814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1
   [junit4]   2> 990816 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 990822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e100cdd{/,null,AVAILABLE}
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4d239dce{HTTP/1.1}{127.0.0.1:54357}
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server Started @993717ms
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=54357, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores, hostContext=/}
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/'
   [junit4]   2> 990825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 990825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 990849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990853 INFO  (zkCallback-2389-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5a7ffa2a name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 990854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/solr.xml
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 981056402
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/]
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/lib
   [junit4]   2> 990869 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/lib).
   [junit4]   2> 990879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 990880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39182/solr
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990882 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990883 INFO  (zkCallback-2391-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@792da9de name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990889 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@30795bfc name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990892 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 990894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 990898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 990901 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 990903 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 990905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 990906 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 990907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 990908 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 990909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54357_
   [junit4]   2> 990909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54357_
   [junit4]   2> 990917 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 990918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 990919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 990920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94883104232767494-127.0.0.1:54357_-n_0000000000
   [junit4]   2> 990921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54357_
   [junit4]   2> 990921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 990923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer Overseer (id=94883104232767494-127.0.0.1:54357_-n_0000000000) starting
   [junit4]   2> 990924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 990932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 990932 INFO  (OverseerCollectionConfigSetProcessor-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 990933 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 990933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 990953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 990954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 990955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores
   [junit4]   2> 990956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/, transient=false, schema=schema.xml, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1, loadOnStartup=true, dataDir=data/, shard=, name=collection1, coreNodeName=, config=solrconfig.xml, collection=control_collection}
   [junit4]   2> 990956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/
   [junit4]   2> 990956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 990957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 990957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 990958 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 990958 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 990959 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 990960 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 990960 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 990961 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 990961 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 990963 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 991961 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 991961 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 991963 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 991963 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/'
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 991984 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 991989 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 992004 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 992034 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 992036 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 992042 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 992317 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 992324 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 992326 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 992336 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 992339 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 992342 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 992343 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 992343 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 992343 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 992344 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 992344 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 992345 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 992345 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 992345 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20603147
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 992346 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/index
   [junit4]   2> 992347 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=43.2099609375, floorSegmentMB=1.296875, forceMergeDeletesPctAllowed=8.243449821918947, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 992347 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5a70997a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166c46f9; maxCacheMB=0.23046875 maxMergeSizeMB=0.6484375)),segFN=segments_1,generation=1}
   [junit4]   2> 992347 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 992348 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/
   [junit4]   2> 992348 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 992352 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 992352 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 992354 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 992358 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992359 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992361 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992362 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992367 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/get,/admin/mbeans,/replication,/admin/segments,/admin/properties,/update/json,/update,/admin/ping,/admin/plugins,/update/json/docs,/update/csv,/admin/system,/dump,/config,/admin/luke,/admin/file,/admin/logging,/admin/threads,/schema
   [junit4]   2> 992369 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 992369 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 992369 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 992370 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 992370 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 992370 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=13.8505859375, floorSegmentMB=1.193359375, forceMergeDeletesPctAllowed=16.90147541995426, segmentsPerTier=2.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21096095258937622
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5a70997a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166c46f9; maxCacheMB=0.23046875 maxMergeSizeMB=0.6484375)),segFN=segments_1,generation=1}
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@35ee41ef[collection1] main
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 992372 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 992372 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 992374 INFO  (searcherExecutor-4227-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@35ee41ef[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518129669867569152
   [junit4]   2> 992376 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518129669867569152
   [junit4]   2> 992377 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 992378 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 992378 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:54357 collection:control_collection shard:shard1
   [junit4]   2> 992380 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 992385 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94883104232767494-core_node1-n_0000000000
   [junit4]   2> 992386 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 992388 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 992388 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 992388 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 992388 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54357/collection1/
   [junit4]   2> 992389 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 992389 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:54357/collection1/ has no replicas
   [junit4]   2> 992390 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 992394 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 992396 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54357/collection1/ shard1
   [junit4]   2> 992399 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 992459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992461 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39b43132 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 992464 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 992465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 992465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992467 INFO  (zkCallback-2395-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@242ee67f name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992472 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 992472 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 992472 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992473 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:54357/collection1/ and leader is http://127.0.0.1:54357/collection1/
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54357
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 992502 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 992502 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 992503 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 992609 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992609 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores/collection1
   [junit4]   2> 992809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001
   [junit4]   2> 992810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@904dd0e{/,null,AVAILABLE}
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@428cfa18{HTTP/1.1}{127.0.0.1:57384}
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server Started @995706ms
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/tempDir-001/jetty1, hostContext=/, hostPort=57384, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores}
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/'
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 992832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992834 INFO  (zkCallback-2396-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5137cbe6 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 992836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/solr.xml
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1156068566
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/]
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/lib
   [junit4]   2> 992851 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/lib).
   [junit4]   2> 992859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 992860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 992860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39182/solr
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992884 INFO  (zkCallback-2398-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34dd3a18 name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992889 INFO  (zkCallback-2399-thread-1-processing-n:127.0.0.1:57384_) [n:127.0.0.1:57384_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ff3b079 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992897 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 993905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57384_
   [junit4]   2> 993905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57384_
   [junit4]   2> 993906 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 993906 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 993906 INFO  (zkCallback-2399-thread-1-processing-n:127.0.0.1:57384_) [n:127.0.0.1:57384_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 993907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 993909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94883104232767499-127.0.0.1:57384_-n_0000000001
   [junit4]   2> 993910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94883104232767494-127.0.0.1:54357_-n_0000000000 to know if I could be the leader
   [junit4]   2> 993923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 993923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 993925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores
   [junit4]   2> 993925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores/collection1, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores/collection1/, name=collection1, collection=collection1, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, transient=false, schema=schema.xml, loadOnStartup=true}
   [junit4]   2> 993926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Luc

[...truncated too long message...]

220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14230, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47944: collection already exists: awholynewstresscollection_collection3_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D89271C806E95B6A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14231, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47944: collection already exists: awholynewstresscollection_collection4_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D89271C806E95B6A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
   [junit4]   2> 1996765 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D89271C806E95B6A]-worker) [n:127.0.0.1:44846_ c:halfdeletedcollection2 s:shard2 r:core_node3 x:halfdeletedcollection2_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001
   [junit4]   2> Nov 17, 2015 11:17:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {multiDefault=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=Memory doPackFST= true), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=FSTOrd50, _version_=PostingsFormat(name=Memory doPackFST= true), id=FSTOrd50, other_tl1=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), a_i1=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene54), range_facet_i_dv=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=es_EC, timezone=America/Mendoza
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=186286808,total=489160704
   [junit4]   2> NOTE: All tests run in this JVM: [TestDistributedGrouping, TestFileDictionaryLookup, SchemaVersionSpecificBehaviorTest, TestDynamicFieldResource, SolrIndexConfigTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, AnalyticsQueryTest, TestMinMaxOnMultiValuedField, BitVectorTest, TestQueryUtils, TestSolrDeletionPolicy2, BasicAuthIntegrationTest, CSVRequestHandlerTest, TestAnalyzeInfixSuggestions, SpellCheckCollatorTest, TestReplicationHandlerBackup, HdfsCollectionsAPIDistributedZkTest, TestSolrConfigHandlerCloud, UpdateRequestProcessorFactoryTest, TestLMJelinekMercerSimilarityFactory, TestSolrCLIRunExample, FileUtilsTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestStandardQParsers, TestManagedSchemaFieldTypeResource, SolrPluginUtilsTest, MinimalSchemaTest, DOMUtilTest, DistributedMLTComponentTest, EnumFieldTest, DocumentBuilderTest, AsyncCallRequestStatusResponseTest, SmileWriterTest, FacetPivotSmallTest, SpatialFilterTest, TestWordDelimiterFilterFactory, TestSimpleQParserPlugin, IndexBasedSpellCheckerTest, UniqFieldsUpdateProcessorFactoryTest, DistributedTermsComponentTest, TestFastLRUCache, TestQuerySenderListener, TestConfigSetsAPIZkFailure, TestBlobHandler, RankQueryTest, TestReRankQParserPlugin, CursorMarkTest, ChaosMonkeySafeLeaderTest, DeleteLastCustomShardedReplicaTest, BJQParserTest, TestImplicitCoreProperties, TermVectorComponentTest, TestExclusionRuleCollectionAccess, TestSchemaResource, ZkCLITest, SpellCheckComponentTest, TestRuleBasedAuthorizationPlugin, TestFieldTypeResource, TestLFUCache, BufferStoreTest, TestSchemaVersionResource, SolrInfoMBeanTest, TestJsonFacets, CoreAdminRequestStatusTest, PreAnalyzedUpdateProcessorTest, TestSolrQueryParserDefaultOperatorResource, SimpleMLTQParserTest, ClusterStateUpdateTest, MBeansHandlerTest, TestTrie, ConvertedLegacyTest, CloudMLTQParserTest, LeaderInitiatedRecoveryOnCommitTest, HighlighterTest, MoreLikeThisHandlerTest, SecurityConfHandlerTest, AssignTest, TestDFRSimilarityFactory, SimplePostToolTest, TermsComponentTest, DocValuesMissingTest, AddBlockUpdateTest, MultiTermTest, TestCloudManagedSchema, TestCustomDocTransformer, TestManagedSchemaFieldResource, TestBinaryResponseWriter, CollectionsAPIDistributedZkTest]
   [junit4] Completed [282/547] on J1 in 1006.98s, 1 test, 1 error <<< FAILURES!

[...truncated 853 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151117_224439_749.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid14910.hprof ...
   [junit4] Heap dump file created [671755793 bytes in 14.849 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 16 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 547 suites (7 ignored), 2164 tests, 1 error, 32 ignored (18 assumptions) [seed: D89271C806E95B6A]

Total time: 219 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1019 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1019/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
329 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=21909, name=qtp1335420383-21909, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=23172, name=Thread-13762, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.SolrCore.checkStale(SolrCore.java:2628)         at org.apache.solr.core.SolrCore.access$300(SolrCore.java:153)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2598)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)    3) Thread[id=19970, name=qtp1335420383-19970-acceptor-0@7d2a04f0-ServerConnector@9fd449{SSL-http/1.1}{127.0.0.1:50058}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=21886, name=qtp989002053-21886, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=22975, name=Thread-13608, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)    6) Thread[id=20225, name=searcherExecutor-3585-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=21568, name=qtp1335420383-21568, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=19937, name=searcherExecutor-3380-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=22169, name=qtp1913862660-22169, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=20460, name=searcherExecutor-3693-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=20147, name=searcherExecutor-3512-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=19950, name=org.eclipse.jetty.server.session.HashSessionManager@5c6d0e69Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=23155, name=zkCallback-2466-thread-30, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=20334, name=searcherExecutor-3635-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=20894, name=qtp322858882-20894, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=20350, name=searcherExecutor-3649-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=21916, name=qtp1913862660-21916, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=23203, name=zkCallback-2505-thread-60, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=19998, name=Scheduler-1279661854, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=22194, name=qtp1335420383-22194, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:732)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=21881, name=qtp1335420383-21881, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=22345, name=searcherExecutor-5100-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=23058, name=Thread-13674, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)   24) Thread[id=23217, name=zkCallback-2458-thread-36-processing-n:127.0.0.1:47961_tq-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   25) Thread[id=22284, name=searcherExecutor-5006-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=20230, name=searcherExecutor-3599-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=22239, name=searcherExecutor-5027-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=22282, name=qtp1335420383-22282, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)         at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1035)         at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1020)         at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:555)         at org.apache.solr.common.cloud.ClusterState.getCollections(ClusterState.java:207)         at org.apache.solr.common.cloud.ZkStateReader.constructState(ZkStateReader.java:461)         at org.apache.solr.common.cloud.ZkStateReader.removeZKWatch(ZkStateReader.java:1103)         at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1308)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:643)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=20237, name=searcherExecutor-3609-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=20302, name=qtp322858882-20302, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:377)         at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:250)         at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1315)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:643)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=22363, name=searcherExecutor-5131-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=22195, name=qtp1335420383-22195, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:732)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=22340, name=searcherExecutor-5086-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=22347, name=searcherExecutor-5099-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=22093, name=searcherExecutor-4952-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=22193, name=qtp989002053-22193, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:732)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=22334, name=searcherExecutor-5064-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=23179, name=zkCallback-2505-thread-44, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=23059, name=Thread-13675, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)   40) Thread[id=22082, name=searcherExecutor-4940-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=22604, name=searcherExecutor-5202-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=19969, name=qtp1335420383-19969-selector-ServerConnectorManager@12c3fb2f/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=23192, name=zkCallback-2466-thread-36, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=20288, name=qtp1335420383-20288, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=23216, name=zkCallback-2458-thread-36-processing-n:127.0.0.1:47961_tq-SendThread(127.0.0.1:33764), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)   46) Thread[id=23063, name=RecoveryThread-awholynewstresscollection_collection0_5_shard9_replica2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:519)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:225)   47) Thread[id=23191, name=zkCallback-2505-thread-51, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=22980, name=Thread-13614, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)   49) Thread[id=22099, name=searcherExecutor-4955-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=19942, name=qtp989002053-19942-selector-ServerConnectorManager@3c6886ac/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=21923, name=qtp1913862660-21923, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=20337, name=searcherExecutor-3636-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=22084, name=searcherExecutor-4950-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=19918, name=qtp1913862660-19918, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=21974, name=qtp322858882-21974, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=19986, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[F01B49D9BE6011D0]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   57) Thread[id=22060, name=searcherExecutor-4911-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=23198, name=zkCallback-2505-thread-55, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=22067, name=searcherExecutor-4923-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=22196, name=qtp322858882-22196, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:73

[...truncated too long message...]

PoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   173) Thread[id=22057, name=searcherExecutor-4904-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   174) Thread[id=23202, name=zkCallback-2505-thread-59, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   175) Thread[id=22201, name=searcherExecutor-5001-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   176) Thread[id=20353, name=searcherExecutor-3652-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   177) Thread[id=22380, name=searcherExecutor-5110-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   178) Thread[id=22358, name=searcherExecutor-5097-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   179) Thread[id=22359, name=searcherExecutor-5108-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   180) Thread[id=23195, name=zkCallback-2505-thread-52, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F01B49D9BE6011D0]:0)
   [junit4] Completed [286/547] on J1 in 1229.44s, 1 test, 3 errors <<< FAILURES!

[...truncated 777 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151116_223518_659.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid32393.hprof ...
   [junit4] Heap dump file created [611051036 bytes in 9.747 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 18 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 547 suites (7 ignored), 2163 tests, 2 suite-level errors, 1 error, 820 ignored (14 assumptions) [seed: F01B49D9BE6011D0]

Total time: 167 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1018 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1018/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([D9C7AF4E9AE9B76C:519390943415DA94]:0)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.getCoreContainer(JettySolrRunner.java:373)
	at org.apache.solr.cloud.ChaosMonkey.expireSession(ChaosMonkey.java:122)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11491 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/init-core-data-001
   [junit4]   2> 3007393 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_jyi/q
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3007427 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3007433 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3007437 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3007450 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_43826_hdfs____.hhgibd/webapp
   [junit4]   2> 3007579 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3007772 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43826
   [junit4]   2> 3007830 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3007832 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3007845 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37143_datanode____.kds499/webapp
   [junit4]   2> 3007965 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3008173 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37143
   [junit4]   2> 3008205 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3008206 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3008220 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40935_datanode____.owpge2/webapp
   [junit4]   2> 3008268 INFO  (IPC Server handler 3 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-0c4ef819-0165-4dbe-bd74-0f73b8bc3238 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd4f7226-de59-4515-9dc7-6c438025e8f7, infoPort=37143, ipcPort=38847, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 3008268 INFO  (IPC Server handler 3 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-426ffcbe-773f-45cd-b671-67cb33db5b82 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd4f7226-de59-4515-9dc7-6c438025e8f7, infoPort=37143, ipcPort=38847, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3008367 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3008599 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40935
   [junit4]   2> 3008680 INFO  (IPC Server handler 8 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-caddcb5b-de83-454d-8270-60cf1eb69806 node DatanodeRegistration(127.0.0.1, datanodeUuid=f75118f0-e840-4aff-bed1-ced4a9456950, infoPort=40935, ipcPort=49651, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3008681 INFO  (IPC Server handler 8 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-9fbb1db4-b323-4fca-8206-f00521017c6b node DatanodeRegistration(127.0.0.1, datanodeUuid=f75118f0-e840-4aff-bed1-ced4a9456950, infoPort=40935, ipcPort=49651, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3008776 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3008776 INFO  (Thread-15984) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3008776 INFO  (Thread-15984) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3008876 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkTestServer start zk server on port:43636
   [junit4]   2> 3008876 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3008877 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3008885 INFO  (zkCallback-3019-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@30329923 name:ZooKeeperConnection Watcher:127.0.0.1:43636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3008885 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3008886 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3008886 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3008890 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3008890 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3008891 INFO  (zkCallback-3020-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1fb5980b name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3008891 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3008892 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3008892 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3008894 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3008896 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3008897 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3008899 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3008899 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3008902 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3008902 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3008904 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3008904 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3008906 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3008906 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3008908 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3008908 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3008910 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3008910 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3008912 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3008912 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3008913 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3008913 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3008915 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3008915 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3008917 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3008917 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3008919 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3008919 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3009212 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1
   [junit4]   2> 3009213 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@557917d5{/_jyi/q,null,AVAILABLE}
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2db1e24f{HTTP/1.1}{127.0.0.1:59584}
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server Started @3011834ms
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:51807/hdfs__localhost_51807__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001_tempDir-002_control_data, hostContext=/_jyi/q, hostPort=59584, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores}
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/'
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3009239 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3009241 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3009242 INFO  (zkCallback-3021-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@795d7d06 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3009242 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3009243 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3009244 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3009244 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/solr.xml
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer New CoreContainer 256502587
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/]
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/lib
   [junit4]   2> 3009269 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/lib).
   [junit4]   2> 3009296 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3009297 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43636/solr
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3009299 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3009300 INFO  (zkCallback-3023-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42c9f8d0 name:ZooKeeperConnection Watcher:127.0.0.1:43636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3009300 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3009301 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3009302 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3009303 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75028438 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3009303 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3009304 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3009306 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3009309 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3009311 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3009313 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3009315 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3009317 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3009318 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3009319 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3009320 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59584__jyi%2Fq
   [junit4]   2> 3009320 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59584__jyi%2Fq
   [junit4]   2> 3009322 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3009323 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3009324 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3009325 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000
   [junit4]   2> 3009325 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59584__jyi%2Fq
   [junit4]   2> 3009325 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3009326 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer Overseer (id=94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) starting
   [junit4]   2> 3009328 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3009334 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3009334 INFO  (OverseerCollectionConfigSetProcessor-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3009335 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3009335 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3009352 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3009352 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3009353 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores
   [junit4]   2> 3009354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, shard=, dataDir=data/, config=solrconfig.xml, configSetProperties=configsetprops.json, schema=schema.xml, transient=false, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/, coreNodeName=, loadOnStartup=true}
   [junit4]   2> 3009354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/
   [junit4]   2> 3009354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3009356 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3009356 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3009356 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3009356 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3009358 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 3009358 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3009358 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3009358 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3009359 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3009361 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3010359 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3010359 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3010360 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3010360 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3010360 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3010361 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/'
   [junit4]   2> 3010361 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3010361 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3010376 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3010382 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3010395 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3010422 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3010423 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3010429 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3010666 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3010673 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3010675 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3010684 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3010686 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3010689 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3010689 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3010690 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3010690 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3010690 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3010690 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3010691 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3010691 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3010691 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:51807/solr_hdfs_home
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f085760
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3010707 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3010708 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3010718 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3010718 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3010726 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 3010726 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 3010726 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 3010733 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 3010733 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3010734 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3010750 INFO  (IPC Server handler 5 on 51807) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38025 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-caddcb5b-de83-454d-8270-60cf1eb69806:NORMAL:127.0.0.1:44092|RBW], ReplicaUnderConstruction[[DISK]DS-0c4ef819-0165-4dbe-bd74-0f73b8bc3238:NORMAL:127.0.0.1:38025|RBW]]} size 0
   [junit4]   2> 3010751 INFO  (IPC Server handler 4 on 51807) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44092 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-0c4ef819-0165-4dbe-bd74-0f73b8bc3238:NORMAL:127.0.0.1:38025|RBW], ReplicaUnderConstruction[[DISK]DS-9fbb1db4-b323-4fca-8206-f00521017c6b:NORMAL:127.0.0.1:44092|FINALIZED]]} size 0
   [junit4]   2> 3010754 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b82e05b9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@31d29ab8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3010754 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3010755 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3010759 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3010759 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3010761 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 3010763 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010764 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010766 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010767 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010771 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/file,/update,/update/json,/admin/segments,/admin/system,/admin/ping,/admin/luke,/replication,/admin/threads,/dump,/update/csv,/get,/admin/logging,/admin/properties,/schema,/config,/admin/plugins,standard,/update/json/docs
   [junit4]   2> 3010773 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3010773 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3010773 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 3010783 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3010783 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3010785 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@34a84739
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b82e05b9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@31d29ab8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@55c5f9d9[collection1] main
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3010792 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3010792 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3010792 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3010794 INFO  (searcherExecutor-6697-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@55c5f9d9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3010795 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:51816/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:51816 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517953666420572160
   [junit4]   2> 3010796 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1517953666420572160
   [junit4]   2> 3010796 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3010796 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3010797 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59584/_jyi/q collection:control_collection shard:shard1
   [junit4]   2> 3010797 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3010800 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94872104019099652-core_node1-n_0000000000
   [junit4]   2> 3010801 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59584/_jyi/q/collection1/
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59584/_jyi/q/collection1/ has no replicas
   [junit4]   2> 3010802 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3010803 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3010805 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3010807 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59584/_jyi/q/collection1/ shard1
   [junit4]   2> 3010808 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3010858 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3010859 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3010860 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@15656154 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3010860 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3010860 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3010860 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3010862 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3010862 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3010863 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3010863 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3010864 INFO  (zkCallback-3027-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25439679 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3010864 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3010864 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3010867 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 3010868 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3010868 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3010868 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:59584/_jyi/q/collection1/ and leader is http://127.0.0.1:59584/_jyi/q/collection1/
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59584/_jyi/q
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3010910 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null} current state version: 2
   [junit4]   2> 3010910 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3010911 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3011015 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3011015 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3011159 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores/collection1
   [junit4]   2> 3011160 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001
   [junit4]   2> 3011160 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3011162 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3997cca9{/_jyi/q,null,AVAILABLE}
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@491f52fd{HTTP/1.1}{127.0.0.1:38016}
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server Started @3013776ms
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:51807/hdfs__localhost_51807__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001_tempDir-002_jetty1, hostPort=38016, hostContext=/_jyi/q, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores}
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/'
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3011180 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3011180 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3011181 INFO  (zkCallback-3028-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56817f5a name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3011182 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3011182 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3011183 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3011183 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/solr.xml
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer New CoreContainer 593435846
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/]
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/lib
   [junit4]   2> 3011197 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/lib).
   [junit4]   2> 3011205 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3011206 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43636/solr
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3011208 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3011209 INFO  (zkCallback-3030-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77658cc4 name:ZooKeeperConnection Watcher:127.0.0.1:43636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3011209 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3011209 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3011211 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3011212 INFO  (zkCallback-3031-thread-1-processing-n:127.0.0.1:38016__jyi%2Fq) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1dd9acd8 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3011212 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3011217 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3012223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38016__jyi%2Fq
   [junit4]   2> 3012223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38016__jyi%2Fq
   [junit4]   2> 3012224 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3012224 INFO  (zkCallback-3031-thread-1-processing-n:127.0.0.1:38016__jyi%2Fq) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3012224 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3012226 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3012227 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94872104019099657-127.0.0.1:38016__jyi%2Fq-n_0000000001
   [junit4]   2> 3012228 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000 to know if I could be the leader
   [junit4]   2> 3012238 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3012238 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3012238 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores
   [juni

[...truncated too long message...]

sorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:829)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 3109051 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [n:127.0.0.1:57352__jyi%2Fq c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001
   [junit4]   2> Nov 16, 2015 12:25:23 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sl, timezone=Africa/Libreville
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=168258384,total=527958016
   [junit4]   2> NOTE: All tests run in this JVM: [RankQueryTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestFieldResource, TestConfigSetImmutable, SpellCheckCollatorTest, LeaderInitiatedRecoveryOnCommitTest, TestEmbeddedSolrServerConstructors, BufferStoreTest, TestExclusionRuleCollectionAccess, TestStressVersions, CursorMarkTest, TestCloudManagedSchema, BJQParserTest, TestJsonFacets, TestDistributedGrouping, TestFileDictionaryLookup, SoftAutoCommitTest, LeaderFailoverAfterPartitionTest, TestDefaultStatsCache, AnalyticsMergeStrategyTest, TermVectorComponentTest, DocValuesTest, MoreLikeThisHandlerTest, MultiTermTest, TestMiniSolrCloudClusterSSL, TestSortingResponseWriter, DeleteLastCustomShardedReplicaTest, ShardRoutingTest, RequestLoggingTest, TestOverriddenPrefixQueryForCustomFieldType, TestSolrDynamicMBean, TestSolrCoreProperties, NotRequiredUniqueKeyTest, SpatialRPTFieldTypeTest, MBeansHandlerTest, SimplePostToolTest, TestJettySolrRunner, TestCodecSupport, TestAddFieldRealTimeGet, AnalyticsQueryTest, SolrIndexSplitterTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestUpdate, SortByFunctionTest, BitVectorTest, TestFastLRUCache, TestBM25SimilarityFactory, SuggesterFSTTest, FileUtilsTest, SuggestComponentTest, TestCustomDocTransformer, TestDFRSimilarityFactory, TestSolrConfigHandlerCloud, TestSolrQueryParserDefaultOperatorResource, TestSurroundQueryParser, ShardRoutingCustomTest, DistributedTermsComponentTest, TestRestManager, CloneFieldUpdateProcessorFactoryTest, TestReRankQParserPlugin, HdfsSyncSliceTest, SolrPluginUtilsTest, ChaosMonkeyNothingIsSafeTest, TestHashQParserPlugin, FacetPivotSmallTest, HdfsDirectoryFactoryTest, BasicFunctionalityTest, HdfsChaosMonkeyNothingIsSafeTest, AddBlockUpdateTest, RestartWhileUpdatingTest, TestLeaderInitiatedRecoveryThread, SmileWriterTest, SecurityConfHandlerTest, FastVectorHighlighterTest, TestCoreContainer, DistanceFunctionTest, ClusterStateUpdateTest, TestStandardQParsers, TestHashPartitioner, SchemaVersionSpecificBehaviorTest, PreAnalyzedUpdateProcessorTest, HighlighterTest, VMParamsZkACLAndCredentialsProvidersTest, DirectSolrSpellCheckerTest, UniqFieldsUpdateProcessorFactoryTest, TestSystemIdResolver, TestStressUserVersions, BasicDistributedZkTest, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, TestFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestRangeQuery, SimpleFacetsTest, TestSolr4Spatial, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, SolrCmdDistributorTest, PeerSyncTest, ConvertedLegacyTest, TestSort, TestFiltering, DirectUpdateHandlerTest, TestIndexSearcher, CurrencyFieldOpenExchangeTest, SignatureUpdateProcessorFactoryTest, TestStressLucene, TestTrie, SpatialFilterTest, TestCSVLoader, WordBreakSolrSpellCheckerTest, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, TestSolrQueryParser, LoggingHandlerTest, ReturnFieldsTest, JSONWriterTest, JsonLoaderTest, PingRequestHandlerTest, HighlighterConfigTest, SOLR749Test, TestQuerySenderListener, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, TestSolrDeletionPolicy2, SampleTest, TestBinaryField, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, ExternalFileFieldSortTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, PluginInfoTest, PreAnalyzedFieldTest, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, ClusterStateTest, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, DistributedMLTComponentTest, TestRTGBase, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, ActionThrottleTest, AssignTest, AsyncCallRequestStatusResponseTest, AsyncMigrateRouteKeyTest, CustomCollectionTest, LeaderInitiatedRecoveryOnShardRestartTest, MigrateRouteKeyTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, SharedFSAutoReplicaFailoverTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestLeaderElectionZkExpiry, TestMiniSolrCloudCluster, TestRequestStatusCollectionAPI, HdfsBasicDistributedZk2Test]
   [junit4] Completed [445/547] on J1 in 111.54s, 1 test, 1 error <<< FAILURES!

[...truncated 355 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 547 suites (7 ignored), 2163 tests, 1 error, 32 ignored (18 assumptions) [seed: D9C7AF4E9AE9B76C]

Total time: 207 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1017 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1017/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:36518/awholynewcollection_4: non ok status: 500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36518/awholynewcollection_4: non ok status: 500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([F58C68861D2356FA:7DD8575CB3DF3B02]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:508)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1753)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:653)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:155)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10975 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid19078.hprof ...
   [junit4] Heap dump file created [638939390 bytes in 9.071 secs]
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.syserr
   [junit4] >>> JVM J0: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J0: EOF ----

[...truncated 559 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/init-core-data-001
   [junit4]   2> 4197256 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 4197256 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4197299 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4197312 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4197328 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4197361 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_36591_hdfs____.3gewek/webapp
   [junit4]   2> 4197495 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4197686 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36591
   [junit4]   2> 4197796 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4197798 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4197811 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_60481_datanode____euxriy/webapp
   [junit4]   2> 4197938 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4198098 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60481
   [junit4]   2> 4198139 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4198142 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4198158 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54118_datanode____.khzyps/webapp
   [junit4]   2> 4198205 INFO  (IPC Server handler 3 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-81243c2d-0bf3-43b9-99a2-f18d7d5a365a node DatanodeRegistration(127.0.0.1, datanodeUuid=ebe56e53-09d0-4832-adb4-537437018b20, infoPort=60481, ipcPort=45674, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4198205 INFO  (IPC Server handler 3 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-8e9d3920-f7ea-4637-b892-ae47e404abe9 node DatanodeRegistration(127.0.0.1, datanodeUuid=ebe56e53-09d0-4832-adb4-537437018b20, infoPort=60481, ipcPort=45674, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4198297 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4198454 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54118
   [junit4]   2> 4198542 INFO  (IPC Server handler 8 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-a7faa45b-4eaa-482f-a43c-18ad15c2b0c5 node DatanodeRegistration(127.0.0.1, datanodeUuid=08a8b00c-65d5-4071-b04e-74f8f22d9d8e, infoPort=54118, ipcPort=42814, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4198543 INFO  (IPC Server handler 8 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-3ddf1adc-19f5-45e5-a5bd-7785777750ad node DatanodeRegistration(127.0.0.1, datanodeUuid=08a8b00c-65d5-4071-b04e-74f8f22d9d8e, infoPort=54118, ipcPort=42814, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4198646 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4198646 INFO  (Thread-59699) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4198646 INFO  (Thread-59699) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4198746 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkTestServer start zk server on port:42590
   [junit4]   2> 4198746 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4198747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4198750 INFO  (zkCallback-2566-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e2f9be5 name:ZooKeeperConnection Watcher:127.0.0.1:42590 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4198750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4198750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4198750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4198753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4198754 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4198755 INFO  (zkCallback-2567-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2da50c8c name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4198755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4198755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4198755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 4198757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 4198758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 4198760 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 4198761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4198761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4198764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4198764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 4198766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4198766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4198767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4198768 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4198769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4198769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4198771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4198771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 4198773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4198773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4198774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4198775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4198776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4198776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4198778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4198778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4198780 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4198780 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4199075 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1
   [junit4]   2> 4199077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4459fa43{/,null,AVAILABLE}
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@28676c43{HTTP/1.1}{127.0.0.1:36938}
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server Started @4201821ms
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=36938, solr.data.dir=hdfs://localhost:60351/hdfs__localhost_60351__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores, hostContext=/}
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 4199080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/'
   [junit4]   2> 4199080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4199080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4199096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4199097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4199098 INFO  (zkCallback-2568-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d42ebd0 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4199098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4199099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4199100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4199100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/solr.xml
   [junit4]   2> 4199114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores
   [junit4]   2> 4199114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer New CoreContainer 799937137
   [junit4]   2> 4199114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/]
   [junit4]   2> 4199115 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/lib
   [junit4]   2> 4199115 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/lib).
   [junit4]   2> 4199123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42590/solr
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4199125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4199125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4199127 INFO  (zkCallback-2570-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c3d3caa name:ZooKeeperConnection Watcher:127.0.0.1:42590 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4199127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4199127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4199128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4199129 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14c02db4 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4199130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4199131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 4199133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 4199136 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 4199138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 4199140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 4199142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 4199144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 4199145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 4199147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 4199148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36938_
   [junit4]   2> 4199149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36938_
   [junit4]   2> 4199150 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 4199152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 4199153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4199154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94866545235394564-127.0.0.1:36938_-n_0000000000
   [junit4]   2> 4199154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36938_
   [junit4]   2> 4199154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 4199156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer Overseer (id=94866545235394564-127.0.0.1:36938_-n_0000000000) starting
   [junit4]   2> 4199158 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 4199166 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4199167 INFO  (OverseerCollectionConfigSetProcessor-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 4199167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4199167 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 4199183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4199183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4199184 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores
   [junit4]   2> 4199185 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/, loadOnStartup=true, coreNodeName=, dataDir=data/, config=solrconfig.xml, shard=, schema=schema.xml, name=collection1, configSetProperties=configsetprops.json, collection=control_collection}
   [junit4]   2> 4199186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/
   [junit4]   2> 4199186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4199187 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 4199187 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4199187 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4199187 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4199190 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 4199190 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4199190 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 4199191 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4199191 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4199193 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4200192 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4200192 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/'
   [junit4]   2> 4200193 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4200193 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4200210 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4200217 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4200284 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 4200313 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4200315 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 4200321 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4200583 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4200591 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 4200593 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 4200600 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4200603 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4200606 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4200606 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4200606 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4200606 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4200607 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4200607 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4200608 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 4200608 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4200608 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:60351/solr_hdfs_home
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7735b56d
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4200627 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4200628 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 4200639 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4200639 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4200686 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4200686 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=0.5712890625, floorSegmentMB=0.4052734375, forceMergeDeletesPctAllowed=18.11247255870828, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 4200721 INFO  (IPC Server handler 7 on 60351) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52142 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-81243c2d-0bf3-43b9-99a2-f18d7d5a365a:NORMAL:127.0.0.1:33094|RBW], ReplicaUnderConstruction[[DISK]DS-3ddf1adc-19f5-45e5-a5bd-7785777750ad:NORMAL:127.0.0.1:52142|RBW]]} size 0
   [junit4]   2> 4200723 INFO  (IPC Server handler 8 on 60351) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33094 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-81243c2d-0bf3-43b9-99a2-f18d7d5a365a:NORMAL:127.0.0.1:33094|RBW], ReplicaUnderConstruction[[DISK]DS-3ddf1adc-19f5-45e5-a5bd-7785777750ad:NORMAL:127.0.0.1:52142|RBW]]} size 0
   [junit4]   2> 4200726 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92c1d99 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@681ee545; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4200726 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4200728 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 4200732 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4200732 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4200734 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4200734 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4200734 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4200734 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 4200737 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200738 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200739 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200741 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200745 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/properties,/get,/admin/logging,/admin/system,/update/json,/admin/segments,/admin/mbeans,/update/json/docs,/admin/file,/update,/admin/ping,/dump,/schema,/admin/threads,/replication,/admin/plugins,/config,/update/csv,/admin/luke
   [junit4]   2> 4200747 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 4200747 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4200747 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 4200760 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 4200760 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 4200761 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=49.2177734375, floorSegmentMB=1.4482421875, forceMergeDeletesPctAllowed=2.931814510927212, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.10836246229761147
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92c1d99 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@681ee545; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2300b23b[collection1] main
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 4200774 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4200775 INFO  (searcherExecutor-8521-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2300b23b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517864725996634112
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1517864725996634112
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 4200776 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 4200776 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:36938 collection:control_collection shard:shard1
   [junit4]   2> 4200777 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 4200781 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94866545235394564-core_node1-n_0000000000
   [junit4]   2> 4200781 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 4200782 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36938/collection1/
   [junit4]   2> 4200783 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36938/collection1/ has no replicas
   [junit4]   2> 4200784 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4200784 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 4200786 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 4200788 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36938/collection1/ shard1
   [junit4]   2> 4200789 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 4200891 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4200939 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:36938/collection1/ and leader is http://127.0.0.1:36938/collection1/
   [junit4]   2> 4200939 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36938
   [junit4]   2> 4200940 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4200940 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 4200940 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4200942 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 4200942 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 4201044 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4201189 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201190 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201191 INFO  (zkCallback-2573-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a659205 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201191 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201191 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201192 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4201194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 4201194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 4201194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201196 INFO  (zkCallback-2574-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@be29249 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201196 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201196 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201198 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x151089ec9d30006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4201198 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 4201199 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 4201301 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 4201491 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores/collection1
   [junit4]   2> 4201491 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001
   [junit4]   2> 4201492 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4201494 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35db059c{/,null,AVAILABLE}
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@49f32389{HTTP/1.1}{127.0.0.1:36518}
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server Started @4204237ms
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores, hostPort=36518, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost:60351/hdfs__localhost_60351__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001_tempDir-002_jetty1, hostContext=/}
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/'
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4201512 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201514 INFO  (zkCallback-2575-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5e711f45 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4201515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/solr.xml
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer New CoreContainer 2136837154
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/]
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/lib
   [junit4]   2> 4201530 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/lib).
   [junit4]   2> 4201538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4201539 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42590/solr
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201543 INFO  (zkCallback-2577-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54983e84 name:ZooKeeperConnection Watcher:127.0.0.1:42590 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201544 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201545 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201547 INFO  (zkCallback-2578-thread-1-processing-n:127.0.0.1:36518_) [n:127.0.0.1:36518_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1be70ed5 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201553 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4202559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36518_
   [junit4]   2> 4202560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36518_
   [junit4]   2> 4202561 INFO  (zkCallback-2571-thread-2-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4202561 INFO  (zkCallback-2573-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4202561 INFO  (zkCallback-2578-thread-1-processing-n:127.0.0.1:36518_) [n:127.0.0.1:36518_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4202562 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4202563 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94866545235394569-127.0.0.1:36518_-n_0000000001
   [junit4]   2> 4202564 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94866545235394564-127.0.0.1:36938_-n_0000000000 to know if I could be the leader
   [junit4]   2> 4202579 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4202579 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4202581 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores
   [junit4]   2> 4202582 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.SolrCore Created CoreDescriptor: 

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 219 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1016 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1016/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.StressHdfsTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:44745/bnc/delete_data_dir

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:44745/bnc/delete_data_dir
	at __randomizedtesting.SeedInfo.seed([F14E7BEDE03608BB:791A44374ECA6543]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:483)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:464)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:197)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 53 more




Build Log:
[...truncated 10402 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.StressHdfsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/init-core-data-001
   [junit4]   2> 491245 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /bnc/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 491318 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 491328 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 491331 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 491354 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_59930_hdfs____.57fuw4/webapp
   [junit4]   2> 491573 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 491955 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59930
   [junit4]   2> 492198 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 492200 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 492221 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_38441_datanode____.fct78t/webapp
   [junit4]   2> 492492 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 492814 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38441
   [junit4]   2> 493050 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 493052 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 493126 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59637_datanode____.kyfdwj/webapp
   [junit4]   2> 493181 INFO  (IPC Server handler 2 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-847ac855-bf43-4298-9ab8-a2efa42ad71a node DatanodeRegistration(127.0.0.1, datanodeUuid=417cd5b5-071e-4cd0-b33b-3a59817f4a13, infoPort=38441, ipcPort=46888, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 493182 INFO  (IPC Server handler 2 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-cc194b65-4d12-47e5-8dff-00028b0d85c2 node DatanodeRegistration(127.0.0.1, datanodeUuid=417cd5b5-071e-4cd0-b33b-3a59817f4a13, infoPort=38441, ipcPort=46888, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 493361 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 493731 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59637
   [junit4]   2> 494005 INFO  (IPC Server handler 8 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-d65f7d23-1798-4bb7-8e48-63fb5dfcaaf2 node DatanodeRegistration(127.0.0.1, datanodeUuid=e51316a5-a6c4-4a5f-9ab2-3e9dbeb260c1, infoPort=59637, ipcPort=54984, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 494005 INFO  (IPC Server handler 8 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-a6552db6-73e4-4410-a10b-c8a5ba21e554 node DatanodeRegistration(127.0.0.1, datanodeUuid=e51316a5-a6c4-4a5f-9ab2-3e9dbeb260c1, infoPort=59637, ipcPort=54984, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 494147 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 494150 INFO  (Thread-1847) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 494150 INFO  (Thread-1847) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 494250 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkTestServer start zk server on port:45773
   [junit4]   2> 494250 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 494257 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 494276 INFO  (zkCallback-144-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d0526f3 name:ZooKeeperConnection Watcher:127.0.0.1:45773 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 494276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 494276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 494276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 494279 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 494282 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 494298 INFO  (zkCallback-145-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7465aad4 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 494298 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 494298 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 494299 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 494301 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 494302 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 494303 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 494304 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 494304 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 494306 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 494307 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 494308 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 494308 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 494309 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 494310 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 494311 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 494311 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 494312 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 494312 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 494314 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 494314 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 494315 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 494315 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 494316 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 494316 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 494318 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 494318 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 494319 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 494319 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 495031 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1
   [junit4]   2> 495033 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 495035 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d2987df{/bnc,null,AVAILABLE}
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@773c3625{HTTP/1.1}{127.0.0.1:39842}
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server Started @498068ms
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores, hostContext=/bnc, hostPort=39842, solr.data.dir=hdfs://localhost:49818/hdfs__localhost_49818__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001_tempDir-002_control_data}
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/'
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 495037 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 495103 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 495105 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495116 INFO  (zkCallback-146-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a531ba0 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495117 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495117 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 495120 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 495120 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/solr.xml
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer New CoreContainer 1574675448
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/]
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/lib
   [junit4]   2> 495141 WARN  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/lib).
   [junit4]   2> 495150 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 495151 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45773/solr
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495154 INFO  (zkCallback-148-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1eb1f18e name:ZooKeeperConnection Watcher:127.0.0.1:45773 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495154 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495155 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 495157 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495165 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6737db10 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495165 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495172 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 495181 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 495189 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 495195 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 495203 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 495207 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 495216 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 495226 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 495230 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 495232 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39842_bnc
   [junit4]   2> 495232 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39842_bnc
   [junit4]   2> 495234 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 495235 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 495237 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 495238 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94861200246046724-127.0.0.1:39842_bnc-n_0000000000
   [junit4]   2> 495238 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39842_bnc
   [junit4]   2> 495238 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 495241 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer Overseer (id=94861200246046724-127.0.0.1:39842_bnc-n_0000000000) starting
   [junit4]   2> 495243 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 495248 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 495249 INFO  (OverseerCollectionConfigSetProcessor-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 495249 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 495249 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 495272 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 495272 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 495273 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores
   [junit4]   2> 495274 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1, coreNodeName=, name=collection1, transient=false, collection=control_collection, dataDir=data/, schema=schema.xml, loadOnStartup=true, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/, configSetProperties=configsetprops.json}
   [junit4]   2> 495274 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/
   [junit4]   2> 495275 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 495276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 495276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 495280 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 495280 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 495285 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 495285 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 495285 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 495288 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 495291 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 495291 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 495291 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 495292 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 495292 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 495292 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 495293 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 495293 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/'
   [junit4]   2> 495293 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 495294 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 495314 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 495322 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 495337 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 495380 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 495382 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 495393 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 496370 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 496390 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 496391 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 496426 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 496440 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 496442 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 496443 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 496443 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 496443 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 496444 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 496444 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49818/solr_hdfs_home
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 496446 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c1dc16a
   [junit4]   2> 496446 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 496479 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 496480 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 496500 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 496502 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 496527 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 496527 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 496527 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 496560 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 496561 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 496561 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=632344122, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 496634 INFO  (IPC Server handler 8 on 49818) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59754 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-d65f7d23-1798-4bb7-8e48-63fb5dfcaaf2:NORMAL:127.0.0.1:38233|RBW], ReplicaUnderConstruction[[DISK]DS-847ac855-bf43-4298-9ab8-a2efa42ad71a:NORMAL:127.0.0.1:59754|RBW]]} size 0
   [junit4]   2> 496638 INFO  (IPC Server handler 9 on 49818) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38233 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-d65f7d23-1798-4bb7-8e48-63fb5dfcaaf2:NORMAL:127.0.0.1:38233|RBW], ReplicaUnderConstruction[[DISK]DS-847ac855-bf43-4298-9ab8-a2efa42ad71a:NORMAL:127.0.0.1:59754|RBW]]} size 0
   [junit4]   2> 496646 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7f2dee8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13d880a6); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 496646 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 496659 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 496661 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 496661 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 496661 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 496661 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 496664 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496667 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496668 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496670 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496672 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 496692 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,/update/json,/replication,/admin/system,/admin/luke,/config,/update/csv,/admin/segments,/admin/threads,/admin/plugins,/update,/admin/file,/admin/mbeans,/dump,/admin/logging,standard,/admin/properties,/schema,/get,/admin/ping
   [junit4]   2> 496694 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 496699 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 496699 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 496737 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 496737 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 496738 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=0.8203125, floorSegmentMB=1.4462890625, forceMergeDeletesPctAllowed=14.568625339501043, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47258112760280724
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7f2dee8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13d880a6); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@691f6645[collection1] main
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 496764 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 496764 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 496765 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 496765 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 496765 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517779206673530880
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1517779206673530880
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 496769 INFO  (searcherExecutor-517-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@691f6645[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 496770 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 496778 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 496797 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:39842/bnc collection:control_collection shard:shard1
   [junit4]   2> 496800 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 496812 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94861200246046724-core_node1-n_0000000000
   [junit4]   2> 496812 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 496815 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 496816 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 496816 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 496819 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@620a97ac name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 496819 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 496819 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 496819 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39842/bnc/collection1/
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39842/bnc/collection1/ has no replicas
   [junit4]   2> 496835 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 496835 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 496843 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 496845 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39842/bnc/collection1/ shard1
   [junit4]   2> 496846 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 496948 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 496948 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 496983 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:39842/bnc/collection1/ and leader is http://127.0.0.1:39842/bnc/collection1/
   [junit4]   2> 496983 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39842/bnc
   [junit4]   2> 496983 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 496984 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 496984 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 496985 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 496986 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 497089 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 497089 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 498391 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1
   [junit4]   2> 498392 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001
   [junit4]   2> 498399 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 498475 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@333c109{/bnc,null,AVAILABLE}
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2b66979a{HTTP/1.1}{127.0.0.1:52555}
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server Started @501508ms
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/bnc, solr.data.dir=hdfs://localhost:49818/hdfs__localhost_49818__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001_tempDir-002_jetty1, hostPort=52555, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/'
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 498592 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 498615 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498616 INFO  (zkCallback-152-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b033f94 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 498616 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 498616 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 498621 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 498621 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/solr.xml
   [junit4]   2> 498654 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores
   [junit4]   2> 498654 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer New CoreContainer 79431362
   [junit4]   2> 498654 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/]
   [junit4]   2> 498655 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/lib
   [junit4]   2> 498655 WARN  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/lib).
   [junit4]   2> 498699 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 498700 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45773/solr
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 498776 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498806 INFO  (zkCallback-154-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f46703a name:ZooKeeperConnection Watcher:127.0.0.1:45773 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 498807 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 498807 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 498818 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498821 INFO  (zkCallback-155-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6422400e name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 498822 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 498828 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 499833 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52555_bnc
   [junit4]   2> 499834 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52555_bnc
   [junit4]   2> 499838 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 499838 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 499839 INFO  (zkCallback-155-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 499840 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 499841 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94861200246046728-127.0.0.1:52555_bnc-n_0000000001
   [junit4]   2> 499842 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94861200246046724-127.0.0.1:39842_bnc-n_0000000000 to know if I could be the leader
   [junit4]   2> 499893 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 499893 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 499894 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores
   [junit4]   2> 499895 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, config=solrconfig.xml, shard=, loadOnStartup=true, coreNodeName=, dataDir=data/, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1/, configSetProperties=configsetprops.json, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1, transient=false}
   [junit4]   2> 499895 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1/
   [junit4]   2> 499895 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 499903 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 499903 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 499906 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 499908 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:52555_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:52555/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 499909 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:52555_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:52555/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 499909 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 499909 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 499917 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 499917 INFO  (zkCallback-155-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 499918 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 499925 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151114_020227_471.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151114_020227_471.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 190 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1015 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1015/

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=3099, name=coreZkRegister-780-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3099, name=coreZkRegister-780-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([345BA5D120782BD9]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10457 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/init-core-data-001
   [junit4]   2> 662956 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[345BA5D120782BD9]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 662962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 662962 INFO  (Thread-625) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 662962 INFO  (Thread-625) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 663062 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkTestServer start zk server on port:49107
   [junit4]   2> 663063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663066 INFO  (zkCallback-885-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@454c2a6d name:ZooKeeperConnection Watcher:127.0.0.1:49107 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663067 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663067 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663067 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 663071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663072 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663074 INFO  (zkCallback-886-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19461ed name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 663077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 663078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 663080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 663082 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 663083 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 663088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 663088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 663091 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663091 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663093 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 663093 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 663095 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 663095 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 663098 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 663098 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 663100 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 663100 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 663103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663107 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 663110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 663111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 663112 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 663429 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1
   [junit4]   2> 663431 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 663434 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b5469f5{/,null,AVAILABLE}
   [junit4]   2> 663436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.ServerConnector Started ServerConnector@e0050{HTTP/1.1}{127.0.0.1:33680}
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server Started @666358ms
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/tempDir-001/control/data, hostPort=33680}
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/'
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 663438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 663456 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663457 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663459 INFO  (zkCallback-887-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65fc9df7 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663475 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 663476 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/solr.xml
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer New CoreContainer 1758502352
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/]
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/lib
   [junit4]   2> 663491 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/lib).
   [junit4]   2> 663500 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49107/solr
   [junit4]   2> 663502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 663502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663504 INFO  (zkCallback-889-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@cf76937 name:ZooKeeperConnection Watcher:127.0.0.1:49107 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663504 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663505 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663511 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1c029382 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663513 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 663516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 663520 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 663525 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 663530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 663532 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 663534 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 663535 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 663537 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 663541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33680_
   [junit4]   2> 663541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33680_
   [junit4]   2> 663545 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 663547 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 663549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 663549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94854745708953604-127.0.0.1:33680_-n_0000000000
   [junit4]   2> 663551 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33680_
   [junit4]   2> 663551 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 663553 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer Overseer (id=94854745708953604-127.0.0.1:33680_-n_0000000000) starting
   [junit4]   2> 663557 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 663570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 663570 INFO  (OverseerCollectionConfigSetProcessor-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 663570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 663571 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 663593 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 663593 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 663594 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores
   [junit4]   2> 663597 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1, schema=schema.xml, collection=control_collection, name=collection1, config=solrconfig.xml, coreNodeName=, dataDir=data/, shard=, configSetProperties=configsetprops.json}
   [junit4]   2> 663597 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/
   [junit4]   2> 663597 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 663599 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 663599 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 663599 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 663600 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 663602 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"} current state version: 0
   [junit4]   2> 663603 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"}
   [junit4]   2> 663603 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 663603 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 663604 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 663605 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/'
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 664622 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 664628 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 664642 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 664671 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 664673 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 664697 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 665013 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 665021 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 665023 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 665033 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 665037 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 665041 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 665044 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 665044 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 665044 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 665045 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 665045 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 665046 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 665046 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 665046 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 665047 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 665047 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4074ca7c
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/index/
   [junit4]   2> 665048 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/index
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3526404574292845]
   [junit4]   2> 665049 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7d03d0ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58e5578),segFN=segments_1,generation=1}
   [junit4]   2> 665049 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 665049 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/
   [junit4]   2> 665050 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 665058 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 665061 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665063 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665065 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665066 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665071 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/update/json/docs,/update/csv,/get,/schema,/config,/admin/system,/admin/threads,/admin/ping,/update/json,/admin/plugins,/admin/segments,/admin/properties,/admin/file,/dump,/update,/admin/luke,/admin/mbeans,/admin/logging,/replication
   [junit4]   2> 665073 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 665073 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 665073 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 665074 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 665074 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.34662299133172625]
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7d03d0ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58e5578),segFN=segments_1,generation=1}
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@75fa6e9[collection1] main
   [junit4]   2> 665076 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 665080 INFO  (searcherExecutor-743-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@75fa6e9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517675933562044416
   [junit4]   2> 665082 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517675933562044416
   [junit4]   2> 665082 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 665083 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 665083 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33680 collection:control_collection shard:shard1
   [junit4]   2> 665084 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 665089 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94854745708953604-core_node1-n_0000000000
   [junit4]   2> 665090 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33680/collection1/
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33680/collection1/ has no replicas
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33680/collection1/ shard1
   [junit4]   2> 665094 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 665094 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 665099 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 665101 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665102 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665104 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33680",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 665104 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59b9bc5 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665104 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 665110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 665110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 665110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665114 INFO  (zkCallback-893-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37d96a49 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665117 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 665117 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 665121 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665121 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33680/collection1/ and leader is http://127.0.0.1:33680/collection1/
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33680
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 665156 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"} current state version: 2
   [junit4]   2> 665157 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"}
   [junit4]   2> 665161 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 665272 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665272 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1
   [junit4]   2> 665463 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001
   [junit4]   2> 665464 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 665467 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@567f2d92{/,null,AVAILABLE}
   [junit4]   2> 665467 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.ServerConnector Started ServerConnector@179a499{HTTP/1.1}{127.0.0.1:33857}
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server Started @668389ms
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/tempDir-001/jetty1, hostPort=33857, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores, hostContext=/}
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/'
   [junit4]   2> 665469 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 665469 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 665490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665492 INFO  (zkCallback-894-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@794e578 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665495 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 665496 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/solr.xml
   [junit4]   2> 665517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores
   [junit4]   2> 665518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer New CoreContainer 812957423
   [junit4]   2> 665518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/]
   [junit4]   2> 665518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/lib
   [junit4]   2> 665518 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/lib).
   [junit4]   2> 665528 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49107/solr
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665531 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665533 INFO  (zkCallback-896-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@601fcce7 name:ZooKeeperConnection Watcher:127.0.0.1:49107 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665534 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665535 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665538 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665541 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11102445 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 666560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33857_
   [junit4]   2> 666560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33857_
   [junit4]   2> 666561 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 666561 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 666562 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 666565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 666568 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94854745708953609-127.0.0.1:33857_-n_0000000001
   [junit4]   2> 666569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94854745708953604-127.0.0.1:33680_-n_0000000000 to know if I could be the leader
   [junit4]   2> 666580 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 666580 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 666584 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores
   [junit4]   2> 666587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, shard=, loadOnStartup=true, schema=schema.xml, transient=false, dataDir=data/, configSetProperties=configsetprops.json, coreNodeName=, config=solrconfig.xml, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1}
   [junit4]   2> 666587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1/
   [junit4]   2> 666587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 666589 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 666590 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 666590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 666590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 666594 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 666594 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 666594 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33857_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33857"} current state version: 3
   [junit4]   2> 666595 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33857_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33857"}
   [junit4]   2> 666595 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 666595 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 666597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 666597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 666699 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 666699 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 666700 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 667597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 667597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1/'
   [junit4]   2> 667599 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 667599 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 667615 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 667622 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 667636 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 667681 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:coll

[...truncated too long message...]

.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:166)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:268)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:450)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1027516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@626bcfa6{HTTP/1.1}{127.0.0.1:32811}
   [junit4]   2> 1027517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@221831c6{/,null,UNAVAILABLE}
   [junit4]   2> 1027519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:32811_ c:all_in_lir s:shard1 r:core_node2 x:all_in_lir_shard1_replica2] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49107 49107
   [junit4]   2> 1027783 INFO  (Thread-625) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49107 49107
   [junit4]   2> 1027785 WARN  (Thread-625) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/collection1/state.json
   [junit4]   2> 	4	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94854745708953604-127.0.0.1:33680_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=345BA5D120782BD9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=da -Dtests.timezone=ART -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    365s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3099, name=coreZkRegister-780-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([345BA5D120782BD9]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1027792 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[345BA5D120782BD9]-worker) [n:127.0.0.1:32811_ c:all_in_lir s:shard1 r:core_node2 x:all_in_lir_shard1_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=da, timezone=ART
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=124801568,total=409468928
   [junit4]   2> NOTE: All tests run in this JVM: [DirectUpdateHandlerOptimizeTest, CoreAdminRequestStatusTest, TestFastWriter, VersionInfoTest, TestObjectReleaseTracker, TestSolrDynamicMBean, SampleTest, TestSolrQueryParserResource, TestCharFilters, TestStressReorder, DocExpirationUpdateProcessorFactoryTest, TestRemoveLastDynamicCopyField, ShardSplitTest, QueryParsingTest, TestRestoreCore, DistributedFacetPivotLongTailTest, TestFileDictionaryLookup, TestCopyFieldCollectionResource, TestDownShardTolerantSearch, ChangedSchemaMergeTest, UpdateParamsTest, TestCloudManagedSchemaConcurrent, ShowFileRequestHandlerTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [109/545] on J2 in 364.87s, 1 test, 1 error <<< FAILURES!

[...truncated 1413 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 1 error, 34 ignored (20 assumptions) [seed: 345BA5D120782BD9]

Total time: 213 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1014 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1014/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15513, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15513, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:35277/pgsc/w: collection already exists: awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)




Build Log:
[...truncated 10593 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/init-core-data-001
   [junit4]   2> 1232913 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B2E09DC85F2096EF]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1232913 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B2E09DC85F2096EF]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /pgsc/w
   [junit4]   2> 1232918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1232918 INFO  (Thread-9565) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1232918 INFO  (Thread-9565) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1233018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkTestServer start zk server on port:45614
   [junit4]   2> 1233018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233022 INFO  (zkCallback-1100-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@339f7f3b name:ZooKeeperConnection Watcher:127.0.0.1:45614 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1233025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233026 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42936b3 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233026 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1233028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1233030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1233031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1233032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1233032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1233035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1233035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1233037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1233038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1233039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1233039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1233041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1233041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1233042 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1233043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1233044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1233044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1233046 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1233046 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1233048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1233048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1233050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1233050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1233051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1233051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1233054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233055 INFO  (zkCallback-1102-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2cdea7bc name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1
   [junit4]   2> 1233355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1233357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2548151d{/pgsc/w,null,AVAILABLE}
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68c9da9e{SSL-http/1.1}{127.0.0.1:43225}
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server Started @1236539ms
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/pgsc/w, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/tempDir-001/control/data, hostPort=43225}
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1233365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/'
   [junit4]   2> 1233365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1233365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1233380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233381 INFO  (zkCallback-1103-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1cb1461 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1233383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/solr.xml
   [junit4]   2> 1233396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores
   [junit4]   2> 1233397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer New CoreContainer 320795723
   [junit4]   2> 1233397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/]
   [junit4]   2> 1233397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/lib
   [junit4]   2> 1233397 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/lib).
   [junit4]   2> 1233405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1233407 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45614/solr
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233411 INFO  (zkCallback-1105-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29822494 name:ZooKeeperConnection Watcher:127.0.0.1:45614 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233414 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e005dcd name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1233416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1233419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1233421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1233422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1233425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1233426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1233427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1233428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1233429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43225_pgsc%2Fw
   [junit4]   2> 1233430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43225_pgsc%2Fw
   [junit4]   2> 1233431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1233432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1233433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1233434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000
   [junit4]   2> 1233434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43225_pgsc%2Fw
   [junit4]   2> 1233434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1233436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer Overseer (id=94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) starting
   [junit4]   2> 1233437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1233443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1233444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1233444 INFO  (OverseerCollectionConfigSetProcessor-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1233444 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1233457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1233457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1233458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores
   [junit4]   2> 1233459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1, coreNodeName=, schema=schema.xml, transient=false, collection=control_collection, loadOnStartup=true, name=collection1, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 1233459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/
   [junit4]   2> 1233459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1233460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1233460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1233460 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1233460 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1233462 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"} current state version: 0
   [junit4]   2> 1233462 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1233462 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"}
   [junit4]   2> 1233463 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1233463 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1233464 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1234462 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1234463 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1234463 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1234463 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/'
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1234477 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1234483 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1234496 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1234522 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1234523 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1234529 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1234752 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1234759 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1234761 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1234770 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1234772 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1234775 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1234775 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1234775 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1234775 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1234776 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1234776 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1234778 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e52e33f
   [junit4]   2> 1234778 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data
   [junit4]   2> 1234778 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1234778 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1234779 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/index
   [junit4]   2> 1234779 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=73.998046875, floorSegmentMB=1.0302734375, forceMergeDeletesPctAllowed=1.1723843984137305, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.605175267747365
   [junit4]   2> 1234781 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ac22fc4; maxCacheMB=0.1884765625 maxMergeSizeMB=0.173828125)),segFN=segments_1,generation=1}
   [junit4]   2> 1234781 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1234781 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/
   [junit4]   2> 1234782 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1234785 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1234785 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1234786 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1234789 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234790 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234791 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234792 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234796 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/admin/threads,/config,/admin/mbeans,/admin/ping,/admin/logging,/update/csv,/admin/system,/admin/file,/admin/plugins,/dump,/replication,/schema,/update/json,standard,/admin/properties,/get,/admin/luke,/update,/update/json/docs
   [junit4]   2> 1234797 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1234798 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1234798 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1234799 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1234799 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1234799 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1855868822, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1234800 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ac22fc4; maxCacheMB=0.1884765625 maxMergeSizeMB=0.173828125)),segFN=segments_1,generation=1}
   [junit4]   2> 1234800 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1234801 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@322c6b38[collection1] main
   [junit4]   2> 1234801 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1234803 INFO  (searcherExecutor-3326-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@322c6b38[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1234804 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1234804 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1234804 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517611820585582592
   [junit4]   2> 1234806 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517611820585582592
   [junit4]   2> 1234806 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1234806 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1234807 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:43225/pgsc/w collection:control_collection shard:shard1
   [junit4]   2> 1234808 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1234811 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94850738663063557-core_node1-n_0000000000
   [junit4]   2> 1234811 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:43225/pgsc/w/collection1/
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:43225/pgsc/w/collection1/ has no replicas
   [junit4]   2> 1234814 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:43225/pgsc/w/collection1/ shard1
   [junit4]   2> 1234814 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1234814 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1234817 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1234819 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1234922 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1234964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1234964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1234966 INFO  (zkCallback-1108-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2226a943 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1234966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1234966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1234966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1234968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1234968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1234968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1234969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1234969 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:43225/pgsc/w/collection1/ and leader is https://127.0.0.1:43225/pgsc/w/collection1/
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:43225/pgsc/w
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1234970 INFO  (zkCallback-1109-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67ccc1b9 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1234971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1234971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1234972 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"} current state version: 2
   [junit4]   2> 1234973 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"}
   [junit4]   2> 1234977 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 1234977 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1234977 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1234980 INFO  (zkCallback-1108-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1235087 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1235274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1
   [junit4]   2> 1235275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001
   [junit4]   2> 1235276 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1235277 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7c393a8f{/pgsc/w,null,AVAILABLE}
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5d0596d2{SSL-http/1.1}{127.0.0.1:35277}
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server Started @1238458ms
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=35277, hostContext=/pgsc/w, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores}
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/'
   [junit4]   2> 1235284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1235284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1235299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1235300 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1235301 INFO  (zkCallback-1110-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b1405e3 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1235301 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1235301 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1235302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1235302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/solr.xml
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer New CoreContainer 1058685286
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/]
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/lib
   [junit4]   2> 1235317 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/lib).
   [junit4]   2> 1235325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1235327 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1235328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1235328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45614/solr
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1235331 INFO  (zkCallback-1112-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@fa76653 name:ZooKeeperConnection Watcher:127.0.0.1:45614 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1235331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1235331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1235332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1235333 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6be6ce05 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1235333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1235339 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1236345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35277_pgsc%2Fw
   [junit4]   2> 1236345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35277_pgsc%2Fw
   [junit4]   2> 1236346 INFO  (zkCallback-1108-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1236346 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1236346 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1236348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1236348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94850738663063562-127.0.0.1:35277_pgsc%2Fw-n_0000000001
   [junit4]   2> 1236349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000 to know if I could be the leader
   [junit4]   2> 1236363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1236363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1236364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores
   [junit4]   2> 1236365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, schema=schema.xml, collection=collection1, transient=false, loadOnStartup=true, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1/, coreNodeName=, configSetProperties=configsetprops.json, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1, config=solrconfig.xml}
   [junit4]   2> 1236365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1/
   [junit4]   2> 1236365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1236366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1236366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1236367 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1236367 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1236368 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1236368 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1236369 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35277_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:35277/pgsc/w"} current state version: 3
   [junit4]   2> 1236369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1236369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1236369 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35277_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:35277/pgsc/w"}
   [junit4]   2> 1236369 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1236370 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1236472 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1236473 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1236473 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1237369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1237369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/tes

[...truncated too long message...]

aused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48443/pgsc/w: collection already exists: awholynewstresscollection_collection2_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15512, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48443/pgsc/w: Could not find collection : awholynewstresscollection_collection3_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15510, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48443/pgsc/w: collection already exists: awholynewstresscollection_collection1_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
   [junit4]   2> 1790963 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B2E09DC85F2096EF]-worker) [n:127.0.0.1:47537_pgsc%2Fw c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=FST50, intDefault=FST50, a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), other_tl1=FST50, id=PostingsFormat(name=Direct), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=PostingsFormat(name=Direct)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=ar_DZ, timezone=Brazil/East
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=111490048,total=510656512
   [junit4]   2> NOTE: All tests run in this JVM: [TestStressVersions, TestWordDelimiterFilterFactory, TestPivotHelperCode, TestPartialUpdateDeduplication, AtomicUpdatesTest, TestDistributedMissingSort, DistributedFacetPivotSmallTest, TestMacros, SaslZkACLProviderTest, DistributedTermsComponentTest, SpellCheckComponentTest, TestSchemaResource, ZkSolrClientTest, HLLUtilTest, TestQuerySenderListener, FastVectorHighlighterTest, IndexBasedSpellCheckerTest, TestRawTransformer, SchemaVersionSpecificBehaviorTest, TestExpandComponent, TestMaxScoreQueryParser, ShardRoutingCustomTest, TestSearcherReuse, TestSolr4Spatial2, HighlighterTest, SpellingQueryConverterTest, SpatialHeatmapFacetsTest, TestFastLRUCache, TestTrieFacet, TestTolerantSearch, NotRequiredUniqueKeyTest, PKIAuthenticationIntegrationTest, TermsComponentTest, SpellCheckCollatorTest, TestClassNameShortening, SolrIndexConfigTest, LoggingHandlerTest, DataDrivenBlockJoinTest, TestRecoveryHdfs, StressHdfsTest, DistributedFacetPivotWhiteBoxTest, ClusterStateUpdateTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestClassicSimilarityFactory, DocValuesTest, TestFreeTextSuggestions, TestQuerySenderNoQuery, TestMiniSolrCloudCluster, TermVectorComponentTest, TestBM25SimilarityFactory, TestDistributedGrouping, TestQueryTypes, ResponseHeaderTest, ChaosMonkeyNothingIsSafeTest, TestTrie, SoftAutoCommitTest, TestUpdate, DistributedQueueTest, OverriddenZkACLAndCredentialsProvidersTest, CSVRequestHandlerTest, TestSerializedLuceneMatchVersion, AnalysisErrorHandlingTest, TestCollapseQParserPlugin, TestConfigSetProperties, MultiTermTest, TestSchemaSimilarityResource, TestAuthenticationFramework, ShardRoutingTest, NumericFieldsTest, TestNonDefinedSimilarityFactory, TestRandomMergePolicy, TestCursorMarkWithoutUniqueKey, DirectSolrSpellCheckerTest, AsyncMigrateRouteKeyTest, SolrTestCaseJ4Test, CoreMergeIndexesAdminHandlerTest, ExitableDirectoryReaderTest, TestCustomSort, SimplePostToolTest, TestClusterStateMutator, TestSystemIdResolver, CollectionStateFormat2Test, ZkStateReaderTest, TestMiniSolrCloudClusterBase, DistributedMLTComponentTest, TestUniqueKeyFieldResource, TestInitQParser, TestManagedResource, SortByFunctionTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [194/545] on J1 in 558.80s, 1 test, 1 error <<< FAILURES!

[...truncated 1135 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 1 error, 34 ignored (20 assumptions) [seed: B2E09DC85F2096EF]

Total time: 221 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1013 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1013/

1 tests failed.
FAILED:  org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test

Error Message:
A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'SerbianNormalizationRegular' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, decimaldigit, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, fingerprint, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenoffset, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, serbiannormalization, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]

Stack Trace:
java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'SerbianNormalizationRegular' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, decimaldigit, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, fingerprint, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenoffset, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, serbiannormalization, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]
	at __randomizedtesting.SeedInfo.seed([150596469108C85C:9D51A99C3FF4A5A4]:0)
	at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122)
	at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42)
	at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:153)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 2814 lines...]
   [junit4] Suite: org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestAllAnalyzersHaveFactories -Dtests.method=test -Dtests.seed=150596469108C85C -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_ZA -Dtests.timezone=America/Argentina/Mendoza -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.88s J0 | TestAllAnalyzersHaveFactories.test <<<
   [junit4]    > Throwable #1: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'SerbianNormalizationRegular' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, decimaldigit, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, fingerprint, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenoffset, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, serbiannormalization, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([150596469108C85C:9D51A99C3FF4A5A4]:0)
   [junit4]    > 	at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122)
   [junit4]    > 	at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:153)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=DefaultSimilarity, locale=en_ZA, timezone=America/Argentina/Mendoza
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=127645408,total=272629760
   [junit4]   2> NOTE: All tests run in this JVM: [TestBrazilianStemFilterFactory, TestGreekLowerCaseFilterFactory, TestPortugueseMinimalStemFilterFactory, ShingleFilterTest, TestScandinavianNormalizationFilter, DateRecognizerSinkTokenizerTest, TestKeepWordFilter, TestHomonyms, NGramTokenFilterTest, TestItalianLightStemFilter, TestWordlistLoader, TestSoraniStemFilterFactory, TestMorph, TestArabicNormalizationFilter, TestHungarianLightStemFilterFactory, TestPortugueseLightStemFilterFactory, TestNorwegianLightStemFilterFactory, TestSingleTokenTokenFilter, TestGalicianMinimalStemFilter, TestHindiAnalyzer, TestTurkishAnalyzer, TestTruncateTokenFilter, TestFrenchAnalyzer, TestGalicianMinimalStemFilterFactory, TestCzechStemmer, TestPersianNormalizationFilter, TestPortugueseStemFilterFactory, TestPortugueseLightStemFilter, TestWikipediaTokenizerFactory, TestPatternTokenizer, EdgeNGramTokenizerTest, TestMappingCharFilterFactory, TestOptionalCondition, TestSerbianNormalizationFilterFactory, NumericPayloadTokenFilterTest, TestKeywordMarkerFilter, TestHindiFilters, TestBulgarianAnalyzer, TestScandinavianFoldingFilter, TestEnglishAnalyzer, TestLimitTokenCountFilter, TestAllAnalyzersHaveFactories]
   [junit4] Completed [100/280] on J0 in 0.90s, 1 test, 1 error <<< FAILURES!

[...truncated 548 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/analysis/build.xml:106: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/analysis/build.xml:38: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 280 suites (2 ignored), 1475 tests, 1 error, 1 ignored [seed: 150596469108C85C]

Total time: 69 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1012 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1012/

All tests passed

Build Log:
[...truncated 4723 lines...]
   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp/junit4-J2-20151109_224734_067.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.security.PrivilegedActionException: java.io.IOException: No space left on device
   [junit4] 	at java.security.AccessController.doPrivileged(Native Method)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:96)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.RunListenerEmitter.testAssumptionFailure(RunListenerEmitter.java:68)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.NoExceptionRunListenerDecorator.testAssumptionFailure(NoExceptionRunListenerDecorator.java:63)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.BeforeAfterRunListenerDecorator.testAssumptionFailure(BeforeAfterRunListenerDecorator.java:69)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier$5.notifyListener(OrderedRunNotifier.java:146)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier$SafeNotifier.run(OrderedRunNotifier.java:63)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier.fireTestAssumptionFailed(OrderedRunNotifier.java:148)
   [junit4] 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$SubNotifier.fireTestAssumptionFailed(ThreadLeakControl.java:175)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:879)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
   [junit4] 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4] 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Caused by: java.io.IOException: No space left on device
   [junit4] 	at java.io.FileOutputStream.writeBytes(Native Method)
   [junit4] 	at java.io.FileOutputStream.write(FileOutputStream.java:345)
   [junit4] 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
   [junit4] 	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.EventsOutputStream.write(EventsOutputStream.java:28)
   [junit4] 	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4] 	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135)
   [junit4] 	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.gson.stream.JsonWriter.string(JsonWriter.java:565)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.gson.stream.JsonWriter.value(JsonWriter.java:414)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.FailureEvent.serialize(FailureEvent.java:46)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer$2.run(Serializer.java:101)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer$2.run(Serializer.java:96)
   [junit4] 	... 29 more
   [junit4] <<< JVM J2: EOF ----

[...truncated 203 lines...]
   [junit4] ERROR: JVM J2 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=AB0208B9DF379D90 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.4.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=5.4.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/J2 -Djunit4.childvm.id=2 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=US-ASCII -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.2.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/classes/java:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp/junit4-J2-20151109_224734_067.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp/junit4-J2-20151109_224734_067.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 134 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1011 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1011/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:43241/awholynewcollection_2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /awholynewcollection_2/select. Reason: <pre>    {trace=java.lang.NullPointerException  at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)  at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  at org.eclipse.jetty.server.Server.handle(Server.java:499)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)  at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43241/awholynewcollection_2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /awholynewcollection_2/select. Reason:
<pre>    {trace=java.lang.NullPointerException
	at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)
	at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E:C35C90D38890C466]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:543)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1754)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:665)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=116988, name=coreZkRegister-4110-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=116988, name=coreZkRegister-4110-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E:9582C1741DCECB6D]:0)
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10394 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/init-core-data-001
   [junit4]   2> 562874 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[4B08AF09266CA99E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 562874 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[4B08AF09266CA99E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 562878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 562878 INFO  (Thread-3802) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 562878 INFO  (Thread-3802) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 562978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkTestServer start zk server on port:41093
   [junit4]   2> 562979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 562979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 562981 INFO  (zkCallback-414-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ea23c19 name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 562982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 562982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 562982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 562985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 562985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 562986 INFO  (zkCallback-415-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2607a393 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 562987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 562987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 562987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 562989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 562990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 562992 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 562993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 562993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 562996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 562996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 562998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 562998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 563000 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 563000 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 563002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 563002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 563004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 563004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 563005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 563005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 563007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 563007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 563008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563009 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 563010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 563012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 563012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 563014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563015 INFO  (zkCallback-416-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3faec4e9 name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 563017 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563019 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54fc1ab2 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 563020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 563022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 563022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 563024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 563024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 563025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 563025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 563027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 563027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 563028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 563029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 563030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 563030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 563032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 563032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 563033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 563035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 563036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 563037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 563038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 563038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 563340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1
   [junit4]   2> 563342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 563343 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f263574{/,null,AVAILABLE}
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@125b11d8{HTTP/1.1}{127.0.0.1:41854}
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server Started @566074ms
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores, hostPort=41854, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/tempDir-001/control/data}
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/'
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 563367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563369 INFO  (zkCallback-418-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37862268 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 563370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/solr.xml
   [junit4]   2> 563400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores
   [junit4]   2> 563400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1772405105
   [junit4]   2> 563401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/]
   [junit4]   2> 563401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/lib
   [junit4]   2> 563401 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/lib).
   [junit4]   2> 563408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 563409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41093/solr
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563412 INFO  (zkCallback-420-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f60ed9f name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563416 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b901df6 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 563419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 563422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 563424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 563427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 563429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 563430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 563432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 563433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 563434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41854_
   [junit4]   2> 563434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41854_
   [junit4]   2> 563436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 563437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 563438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 563439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94832189490528262-127.0.0.1:41854_-n_0000000000
   [junit4]   2> 563439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41854_
   [junit4]   2> 563439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 563441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer Overseer (id=94832189490528262-127.0.0.1:41854_-n_0000000000) starting
   [junit4]   2> 563442 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 563448 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 563449 INFO  (OverseerCollectionConfigSetProcessor-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 563449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 563449 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 563465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 563466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 563466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores
   [junit4]   2> 563467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, collection=control_collection, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/, schema=schema.xml, shard=, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1, config=solrconfig.xml, name=collection1, transient=false, loadOnStartup=true}
   [junit4]   2> 563467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/
   [junit4]   2> 563467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 563468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 563468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 563468 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 563468 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 563471 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 563471 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 563471 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 563471 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 563472 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 563473 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 564471 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 564471 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 564472 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 564472 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/'
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 564488 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 564499 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 564516 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 564550 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 564572 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 564580 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 565031 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 565039 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 565045 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 565054 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 565057 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 565060 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 565061 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 565062 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 565062 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 565062 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@28ee7d61
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/index/
   [junit4]   2> 565063 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 565064 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/index
   [junit4]   2> 565064 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=19.6376953125, floorSegmentMB=1.408203125, forceMergeDeletesPctAllowed=9.387081741303945, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.3232421875, noCFSRatio=0.812134224251013
   [junit4]   2> 565065 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@3be62ea0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ff2919; maxCacheMB=0.8740234375 maxMergeSizeMB=0.294921875)),segFN=segments_1,generation=1}
   [junit4]   2> 565065 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 565069 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 565069 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 565071 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 565074 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/
   [junit4]   2> 565074 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 565076 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565078 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565079 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565080 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565084 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/get,/replication,/config,/schema,/admin/file,standard,/admin/threads,/admin/system,/admin/plugins,/admin/luke,/admin/ping,/admin/properties,/update/json,/update/json/docs,/admin/logging,/admin/mbeans,/dump,/admin/segments,/update
   [junit4]   2> 565085 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 565086 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 565086 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20798483476383636]
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@3be62ea0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ff2919; maxCacheMB=0.8740234375 maxMergeSizeMB=0.294921875)),segFN=segments_1,generation=1}
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@71ef6bc0[collection1] main
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 565090 INFO  (searcherExecutor-1629-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@71ef6bc0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517315034164756480
   [junit4]   2> 565092 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517315034164756480
   [junit4]   2> 565093 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 565093 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 565101 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:41854 collection:control_collection shard:shard1
   [junit4]   2> 565103 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 565106 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94832189490528262-core_node1-n_0000000000
   [junit4]   2> 565107 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 565108 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 565108 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 565109 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 565109 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41854/collection1/
   [junit4]   2> 565109 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 565110 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:41854/collection1/ has no replicas
   [junit4]   2> 565110 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41854/collection1/ shard1
   [junit4]   2> 565110 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 565120 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 565122 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 565124 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 565125 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 565142 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 565256 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:41854/collection1/ and leader is http://127.0.0.1:41854/collection1/
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41854
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 565298 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 565299 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 565402 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 565470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 565471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565472 INFO  (zkCallback-423-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2df15bb2 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 565472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 565472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 565472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 565474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 565772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/cores/collection1
   [junit4]   2> 565773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001
   [junit4]   2> 565774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 565776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e8ca006{/,null,AVAILABLE}
   [junit4]   2> 565776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3591ef75{HTTP/1.1}{127.0.0.1:60699}
   [junit4]   2> 565776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server Started @568503ms
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/tempDir-001/jetty1, hostPort=60699, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/cores}
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/'
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 565793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 565794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565795 INFO  (zkCallback-424-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e4acdf0 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 565795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 565795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 565796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 565796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/solr.xml
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/cores
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1192663454
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/]
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/lib
   [junit4]   2> 565810 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/lib).
   [junit4]   2> 565818 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 565819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 565819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41093/solr
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565822 INFO  (zkCallback-426-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f7174fb name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 565822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 565823 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 565824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:60699_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565825 INFO  (zkCallback-427-thread-1-processing-n:127.0.0.1:60699_) [n:127.0.0.1:60699_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.c

[...truncated too long message...]

at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:86)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:169)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:286)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:468)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2416914 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@59a95e4c{HTTP/1.1}{127.0.0.1:58129}
   [junit4]   2> 2416914 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@18fa503b{/,null,UNAVAILABLE}
   [junit4]   2> 2416917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [n:127.0.0.1:58129_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ChaosMonkey monkey: stop shard! 46684
   [junit4]   2> 2416918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [n:127.0.0.1:58129_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59074 59074
   [junit4]   2> 2417154 INFO  (Thread-109844) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59074 59074
   [junit4]   2> 2417154 WARN  (Thread-109844) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	18	/solr/collections/collection1/state.json
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94832308381155332-127.0.0.1:46684_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=4B08AF09266CA99E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_QA -Dtests.timezone=Etc/Universal -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   40.2s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=116988, name=coreZkRegister-4110-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E:9582C1741DCECB6D]:0)
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2417159 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[4B08AF09266CA99E]-worker) [n:127.0.0.1:58129_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=ar_QA, timezone=Etc/Universal
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=275076840,total=520093696
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedExpandComponentTest, TestHashQParserPlugin, SolrCoreCheckLockOnStartupTest, ClusterStateTest, TestSolrQueryParserResource, TestPHPSerializedResponseWriter, ChangedSchemaMergeTest, TestReload, TermVectorComponentDistributedTest, AsyncCallRequestStatusResponseTest, TestRuleBasedAuthorizationPlugin, RollingRestartTest, LukeRequestHandlerTest, TestSchemaVersionResource, TestManagedSchemaFieldTypeResource, StatsComponentTest, SecurityConfHandlerTest, BufferStoreTest, SliceStateTest, TestBlendedInfixSuggestions, TestRestManager, TestSolrDynamicMBean, TestStressRecovery, SuggesterTSTTest, CursorPagingTest, TestIndexSearcher, FieldAnalysisRequestHandlerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DocExpirationUpdateProcessorFactoryTest, TestSearchPerf, HdfsBasicDistributedZkTest, TestDefaultSearchFieldResource, TestSmileRequest, TestRemoveLastDynamicCopyField, TestCollationField, HdfsLockFactoryTest, UpdateParamsTest, DistributedFacetPivotLongTailTest, TestIntervalFaceting, TestDynamicFieldCollectionResource, BJQParserTest, QueryResultKeyTest, TestStressReorder, TestReloadAndDeleteDocs, TestSolrQueryParserDefaultOperatorResource, TestSort, TestCopyFieldCollectionResource, TestFuzzyAnalyzedSuggestions, TestConfigSetImmutable, AssignTest, TestCloudManagedSchema, SimpleFacetsTest, TestReversedWildcardFilterFactory, RegexBoostProcessorTest, RequestHandlersTest, SearchHandlerTest, TestCoreDiscovery, TestSolrQueryParser, TestDefaultStatsCache, TestSweetSpotSimilarityFactory, TestFieldTypeResource, TestGroupingSearch, TestPerFieldSimilarity, TestRecovery, CoreAdminCreateDiscoverTest, CoreAdminRequestStatusTest, TestUtils, TestOverriddenPrefixQueryForCustomFieldType, CachingDirectoryFactoryTest, TestIndexingPerformance, ScriptEngineTest, TestSolrIndexConfig, ShowFileRequestHandlerTest, TestSchemaManager, TestCloudManagedSchemaConcurrent, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [366/545] on J2 in 40.75s, 1 test, 1 error <<< FAILURES!

[...truncated 595 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 2 errors, 32 ignored (18 assumptions) [seed: 4B08AF09266CA99E]

Total time: 193 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1010 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1010/

3 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
Expected that the LIR state would've been down by now expected:<down> but was:<recovering>

Stack Trace:
java.lang.AssertionError: Expected that the LIR state would've been down by now expected:<down> but was:<recovering>
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3:FF9A679486E5BC2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.apache.solr.cloud.ForceLeaderTest.putNonLeadersIntoLIR(ForceLeaderTest.java:386)
	at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:92)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.OverseerTest.testOverseerStatsReset

Error Message:
expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<0> but was:<1>
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3:423A7585E43601AD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:732)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=160214, name=coreZkRegister-966-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=160214, name=coreZkRegister-966-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3:37E4FCC44A4EC050]:0)
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10620 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/init-core-data-001
   [junit4]   2> 1477012 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[E96E92B971ECA2A3]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1477022 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1477023 INFO  (Thread-158410) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1477023 INFO  (Thread-158410) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1477123 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkTestServer start zk server on port:36301
   [junit4]   2> 1477123 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477125 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477140 INFO  (zkCallback-298-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2098723 name:ZooKeeperConnection Watcher:127.0.0.1:36301 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477140 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1477163 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150e44979190000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1477164 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477172 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477182 INFO  (zkCallback-299-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1469e01e name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477182 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477183 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477183 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1477188 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1477189 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1477192 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1477194 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1477194 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1477199 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1477199 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1477205 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1477205 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1477210 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1477211 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1477214 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1477214 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1477217 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1477217 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1477222 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1477222 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1477225 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1477226 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1477229 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1477229 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1477232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1477232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1477238 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1477238 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1477590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1
   [junit4]   2> 1477592 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1477595 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6abd1abc{/,null,AVAILABLE}
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7b470ba2{HTTP/1.1}{127.0.0.1:35300}
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server Started @1480714ms
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/tempDir-001/control/data, hostContext=/, hostPort=35300}
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/'
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1477614 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477616 INFO  (zkCallback-300-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42e5e1f7 name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477616 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477616 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1477617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/solr.xml
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer New CoreContainer 1538089409
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/]
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/lib
   [junit4]   2> 1477632 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/lib).
   [junit4]   2> 1477642 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1477643 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1477643 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1477643 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1477644 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36301/solr
   [junit4]   2> 1477644 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1477644 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477652 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477663 INFO  (zkCallback-302-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5908822f name:ZooKeeperConnection Watcher:127.0.0.1:36301 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477664 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477665 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477670 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477675 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@361956ce name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477675 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477677 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1477682 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1477691 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1477703 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1477710 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1477713 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1477715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1477716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1477718 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1477720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35300_
   [junit4]   2> 1477721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35300_
   [junit4]   2> 1477723 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1477734 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1477744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1477744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94826596390207492-127.0.0.1:35300_-n_0000000000
   [junit4]   2> 1477745 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35300_
   [junit4]   2> 1477745 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1477746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer Overseer (id=94826596390207492-127.0.0.1:35300_-n_0000000000) starting
   [junit4]   2> 1477750 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1477766 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1477767 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1477767 INFO  (OverseerCollectionConfigSetProcessor-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1477767 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1477806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1477806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1477808 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores
   [junit4]   2> 1477808 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1, schema=schema.xml, configSetProperties=configsetprops.json, shard=, transient=false, dataDir=data/, config=solrconfig.xml, name=collection1, loadOnStartup=true}
   [junit4]   2> 1477809 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/
   [junit4]   2> 1477809 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1477811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1477811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1477813 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1477813 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1477818 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1477818 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1477819 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1477820 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1477821 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1477825 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1478819 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1478819 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/'
   [junit4]   2> 1478821 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1478821 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1478840 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1478848 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1478872 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1478925 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1478926 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1478934 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1479221 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1479241 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1479242 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1479258 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1479262 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1479266 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1479267 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1479267 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1479267 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1479268 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1479268 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44e8518c
   [junit4]   2> 1479270 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data
   [junit4]   2> 1479270 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1479270 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1479271 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/index
   [junit4]   2> 1479271 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=88.4921875, floorSegmentMB=0.4912109375, forceMergeDeletesPctAllowed=5.312305620847431, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1479273 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cdea4c1),segFN=segments_1,generation=1}
   [junit4]   2> 1479273 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1479278 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1479278 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1479280 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1479283 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479284 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479286 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479286 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/
   [junit4]   2> 1479287 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1479301 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479307 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/ping,/admin/threads,/replication,/dump,/update,/admin/file,/admin/logging,/config,/update/json,/admin/system,/admin/segments,/admin/plugins,/admin/properties,/update/csv,/admin/luke,/admin/mbeans,/schema,/update/json/docs,/get,standard
   [junit4]   2> 1479309 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1479310 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1479310 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1479311 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1479311 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1479312 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=15, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cdea4c1),segFN=segments_1,generation=1}
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@32e1b33c[collection1] main
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1479322 INFO  (searcherExecutor-929-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@32e1b33c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1479322 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1479322 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1479323 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1479323 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517225544652947456
   [junit4]   2> 1479327 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 4.0ms to seed version buckets with highest version 1517225544652947456
   [junit4]   2> 1479327 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1479329 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1479332 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:35300 collection:control_collection shard:shard1
   [junit4]   2> 1479333 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1479342 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94826596390207492-core_node1-n_0000000000
   [junit4]   2> 1479343 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1479347 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35300/collection1/
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35300/collection1/ has no replicas
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35300/collection1/ shard1
   [junit4]   2> 1479351 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1479356 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1479359 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1479364 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 1479365 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1479371 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 1479474 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:35300/collection1/ and leader is http://127.0.0.1:35300/collection1/
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35300
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1479520 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 1479521 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1479624 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1479813 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1479814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1479820 INFO  (zkCallback-305-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@448af322 name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1479820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1479820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1479820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1479823 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1479823 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1479823 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1479824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1479828 INFO  (zkCallback-306-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@676f09ed name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1479828 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1479828 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1479831 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 1479832 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1479938 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1480409 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/cores/collection1
   [junit4]   2> 1480410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001
   [junit4]   2> 1480416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@cc2c7c7{/,null,AVAILABLE}
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@47c9e05c{HTTP/1.1}{127.0.0.1:42516}
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server Started @1483565ms
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/cores, hostContext=/, hostPort=42516, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/tempDir-001/jetty1}
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/'
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1480484 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1480485 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1480487 INFO  (zkCallback-307-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d1dbab2 name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1480488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1480488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1480489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1480489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/solr.xml
   [junit4]   2> 1480507 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/cores
   [junit4]   2> 1480508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer New CoreContainer 1032188313
   [junit4]   2> 1480508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/]
   [junit4]   2> 1480508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/lib
   [junit4]   2> 1480508 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/lib).
   [junit4]   2> 1480539 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1480543 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36301/solr
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1480545 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1480559 INFO  (zkCallback-309-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@264c8074 name:ZooKeeperConnection Watcher:127.0.0.1:36301 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1480560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151107_230551_577.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151107_230551_577.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 210 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1009 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1009/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=10145, name=coreZkRegister-1917-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=10145, name=coreZkRegister-1917-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818:6FACDD5E4BF6BAEB]:0)
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=533, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=533, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818:39728CF9DEA8B5E0]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55434/_t/px: Could not find collection : awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10131 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B126B3237054D818]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 26   INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B126B3237054D818]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_t/px
   [junit4]   2> 73   INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 76   INFO  (Thread-1) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 76   INFO  (Thread-1) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 175  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkTestServer start zk server on port:35233
   [junit4]   2> 234  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 434  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498  INFO  (zkCallback-1-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@661cd479 name:ZooKeeperConnection Watcher:127.0.0.1:35233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 499  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 505  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 537  WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150df45dd520000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 538  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 544  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 545  INFO  (zkCallback-2-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2541b7cd name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 546  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 546  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 553  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 563  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 569  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 573  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 580  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 595  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 605  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 606  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 613  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 614  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 619  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 620  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 626  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 628  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 643  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 644  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 650  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 652  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 657  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 658  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 664  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 664  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 672  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 673  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 678  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 679  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1
   [junit4]   2> 1910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.u.log Logging initialized @5125ms
   [junit4]   2> 2083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d0b4c11{/_t/px,null,AVAILABLE}
   [junit4]   2> 2179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.ServerConnector Started ServerConnector@13d10cf2{HTTP/1.1}{127.0.0.1:57881}
   [junit4]   2> 2179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server Started @5395ms
   [junit4]   2> 2180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_t/px, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/tempDir-001/control/data, hostPort=57881, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores}
   [junit4]   2> 2189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2235 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/'
   [junit4]   2> 2240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2424 INFO  (zkCallback-3-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@696f551d name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/solr.xml
   [junit4]   2> 2585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores
   [junit4]   2> 2645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer New CoreContainer 98711131
   [junit4]   2> 2645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/]
   [junit4]   2> 2645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/lib
   [junit4]   2> 2646 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/lib).
   [junit4]   2> 2693 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3045 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3052 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35233/solr
   [junit4]   2> 3070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3073 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3081 INFO  (zkCallback-5-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ba240e5 name:ZooKeeperConnection Watcher:127.0.0.1:35233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3082 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3082 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3098 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6320932f name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3098 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3110 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57881__t%2Fpx
   [junit4]   2> 3196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57881__t%2Fpx
   [junit4]   2> 3204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3214 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000
   [junit4]   2> 3223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57881__t%2Fpx
   [junit4]   2> 3223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer Overseer (id=94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) starting
   [junit4]   2> 3244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3413 INFO  (OverseerCollectionConfigSetProcessor-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3474 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores
   [junit4]   2> 4071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/, dataDir=data/, transient=false, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1, schema=schema.xml, configSetProperties=configsetprops.json, loadOnStartup=true, collection=control_collection, name=collection1, coreNodeName=}
   [junit4]   2> 4072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/
   [junit4]   2> 4072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4092 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4092 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 4095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4107 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 4110 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4118 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 4121 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4142 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4150 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 5115 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 5115 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 5116 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 5116 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 5118 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 5120 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/'
   [junit4]   2> 5121 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 5121 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 5151 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 5194 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 5327 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 5570 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 5580 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 5789 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 6712 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 6805 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 6833 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 7111 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 7129 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 7136 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 7140 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 7140 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 7140 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 7146 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 7146 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 7151 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 7152 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 7153 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 7168 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 7172 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b557441
   [junit4]   2> 7247 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data
   [junit4]   2> 7250 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/index/
   [junit4]   2> 7250 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 7264 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/index
   [junit4]   2> 7266 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=67.0009765625, floorSegmentMB=0.3671875, forceMergeDeletesPctAllowed=9.749867699448995, segmentsPerTier=25.0, maxCFSSegmentSizeMB=1.27734375, noCFSRatio=0.0
   [junit4]   2> 7332 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@31ec634b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166aea88),segFN=segments_1,generation=1}
   [junit4]   2> 7333 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 7346 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/
   [junit4]   2> 7346 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 7629 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 7651 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 7656 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 7656 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 7656 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 7657 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 7679 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 7680 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 7680 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 7702 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 7744 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7752 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7755 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7757 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7803 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/replication,/admin/threads,/update,/admin/luke,/admin/logging,/get,/admin/system,/admin/ping,/dump,/admin/mbeans,/admin/file,/update/csv,standard,/update/json,/config,/admin/segments,/admin/plugins,/admin/properties,/update/json/docs
   [junit4]   2> 7806 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 7829 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 7829 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 7841 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 7841 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 7843 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=1.921875, floorSegmentMB=1.025390625, forceMergeDeletesPctAllowed=12.156630515120758, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 7849 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@31ec634b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166aea88),segFN=segments_1,generation=1}
   [junit4]   2> 7849 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 7870 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@25e3ff47[collection1] main
   [junit4]   2> 7881 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 7883 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 7883 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 7887 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 7887 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 7908 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 7910 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 7910 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 7910 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 7936 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 7941 INFO  (searcherExecutor-8-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@25e3ff47[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 7941 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 7943 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 7943 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 7943 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517137341494329344
   [junit4]   2> 7948 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 5.0ms to seed version buckets with highest version 1517137341494329344
   [junit4]   2> 7950 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 7952 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 7953 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57881/_t/px collection:control_collection shard:shard1
   [junit4]   2> 7961 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 7968 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94821083333656580-core_node1-n_0000000000
   [junit4]   2> 7970 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 7974 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 7974 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 7975 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 7976 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57881/_t/px/collection1/
   [junit4]   2> 7976 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 7977 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57881/_t/px/collection1/ has no replicas
   [junit4]   2> 7977 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57881/_t/px/collection1/ shard1
   [junit4]   2> 7980 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 7982 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 8001 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 8022 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 8024 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 8029 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 8124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8127 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ce7372b name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 8134 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8134 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 8135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 8135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8139 INFO  (zkCallback-9-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@dfa0111 name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8146 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150df45dd520006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 8146 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 8147 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 8178 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57881/_t/px/collection1/ and leader is http://127.0.0.1:57881/_t/px/collection1/
   [junit4]   2> 8179 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57881/_t/px
   [junit4]   2> 8180 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 8180 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 8180 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 8184 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2"} current state version: 3
   [junit4]   2> 8193 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 8196 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 8306 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8306 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1
   [junit4]   2> 8495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001
   [junit4]   2> 8496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 8502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5504f2ca{/_t/px,null,AVAILABLE}
   [junit4]   2> 8503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.ServerConnector Started ServerConnector@48b53def{HTTP/1.1}{127.0.0.1:39114}
   [junit4]   2> 8504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server Started @11719ms
   [junit4]   2> 8504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=39114, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_t/px}
   [junit4]   2> 8505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 8505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/'
   [junit4]   2> 8506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 8506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 8535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8538 INFO  (zkCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43fb37ae name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 8542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/solr.xml
   [junit4]   2> 8571 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores
   [junit4]   2> 8572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer New CoreContainer 1837657652
   [junit4]   2> 8572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/]
   [junit4]   2> 8572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/lib
   [junit4]   2> 8572 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/lib).
   [junit4]   2> 8586 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 8588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 8589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35233/solr
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8607 INFO  (zkCallback-12-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@28ad11e6 name:ZooKeeperConnection Watcher:127.0.0.1:35233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8619 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@650f5b42 name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 9659 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39114__t%2Fpx
   [junit4]   2> 9662 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39114__t%2Fpx
   [junit4]   2> 9665 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 9665 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 9665 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 9670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 9672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94821083333656585-127.0.0.1:39114__t%2Fpx-n_0000000001
   [junit4]   2> 9676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000 to know if I could be the leader
   [junit4]   2> 9717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 9717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 9719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores
   [junit4]   2> 9720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, transient=false, collection=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1/, schema=schema.xml, dataDir=data/, configSetProperties=configsetprops.json, shard=, loadOnStartup=true, name=collection1, config=solrconfig.xml}
   [junit4]   2> 9720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1/
   [junit4]   2> 9721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 9722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 9723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 9725 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 9725 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 9730 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 9730 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 9732 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:39114__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:39114/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2"} current state version: 4
   [junit4]   2> 9733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 9733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 9733 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:39114__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:39114/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 9734 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 9734 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 9841 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 9843 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 9844 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 10733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 10733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 10735 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 10735 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 10736 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 10736 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1/'
   [junit4]   2> 10737 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 10737 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 10760 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 10787 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 10821 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 10873 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 10875 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 10980 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 11533 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 11554 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 11560 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.

[...truncated too long message...]

Client$2.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:86)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:169)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:286)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:468)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 933410 WARN  (zkCallback-690-thread-1-processing-n:127.0.0.1:57305_) [n:127.0.0.1:57305_    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 933411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@66cee64d{HTTP/1.1}{127.0.0.1:57305}
   [junit4]   2> 933411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@526f817c{/,null,UNAVAILABLE}
   [junit4]   2> 933418 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [n:127.0.0.1:57305_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ChaosMonkey monkey: stop shard! 40100
   [junit4]   2> 933419 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [n:127.0.0.1:57305_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34222 34222
   [junit4]   2> 933688 INFO  (Thread-6297) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34222 34222
   [junit4]   2> 933689 WARN  (Thread-6297) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/collection1/state.json
   [junit4]   2> 	3	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94821141467299844-127.0.0.1:40100_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=B126B3237054D818 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en -Dtests.timezone=Africa/Bissau -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   46.4s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=10145, name=coreZkRegister-1917-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B126B3237054D818:6FACDD5E4BF6BAEB]:0)
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B126B3237054D818]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 933692 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[B126B3237054D818]-worker) [n:127.0.0.1:57305_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=en, timezone=Africa/Bissau
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=262534528,total=414187520
   [junit4]   2> NOTE: All tests run in this JVM: [TestAddFieldRealTimeGet, RequestHandlersTest, HttpPartitionTest, ChangedSchemaMergeTest, PrimitiveFieldTypeTest, TestElisionMultitermQuery, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestIndexingPerformance, SimpleFacetsTest, TestJettySolrRunner, TestFieldTypeResource, StatsComponentTest, SecurityConfHandlerTest, TestExceedMaxTermLength, TestDefaultSearchFieldResource, ZkStateWriterTest, TestCSVResponseWriter, BadIndexSchemaTest, LeaderElectionTest, TestCollationField, SmileWriterTest, RankQueryTest, SliceStateTest, AssignTest, DirectUpdateHandlerTest, TestSchemaVersionResource, HardAutoCommitTest, TestManagedSchemaFieldTypeResource, CurrencyFieldXmlFileTest, TestSmileRequest, HighlighterConfigTest, BlockCacheTest, TestShardHandlerFactory, TestUtils, RecoveryZkTest, TestCopyFieldCollectionResource, TestAnalyzeInfixSuggestions, CursorPagingTest, QueryParsingTest, HdfsSyncSliceTest, OverseerTest, PluginInfoTest, TestReload, TestSolrQueryParserDefaultOperatorResource, QueryResultKeyTest, TestBlobHandler, TestRemoveLastDynamicCopyField, DirectUpdateHandlerOptimizeTest, TestZkChroot, TermVectorComponentDistributedTest, TestLeaderInitiatedRecoveryThread, SuggesterTSTTest, CustomCollectionTest, TestSolrCloudWithKerberosAlt, FieldAnalysisRequestHandlerTest, ShowFileRequestHandlerTest, DocExpirationUpdateProcessorFactoryTest, CoreAdminCreateDiscoverTest, TestObjectReleaseTracker, TestGroupingSearch, TestSolrQueryParserResource, DocValuesMultiTest, ScriptEngineTest, TestDownShardTolerantSearch, UpdateParamsTest, TestDistribIDF, FacetPivotSmallTest, TestFuzzyAnalyzedSuggestions, SolrCoreCheckLockOnStartupTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [151/545] on J2 in 46.79s, 1 test, 1 error <<< FAILURES!

[...truncated 1297 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 2 errors, 32 ignored (18 assumptions) [seed: B126B3237054D818]

Total time: 189 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1008 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1008/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:39301/_aki/awholynewcollection_1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /_aki/awholynewcollection_1/select. Reason: <pre>    {trace=java.lang.NullPointerException  at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)  at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  at org.eclipse.jetty.server.Server.handle(Server.java:499)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)  at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39301/_aki/awholynewcollection_1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /_aki/awholynewcollection_1/select. Reason:
<pre>    {trace=java.lang.NullPointerException
	at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)
	at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35:F28510E717CED2CD]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:543)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1754)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:665)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=14042, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=14042, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)




Build Log:
[...truncated 10856 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/init-core-data-001
   [junit4]   2> 1432007 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7AD12F3DB932BF35]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1432007 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7AD12F3DB932BF35]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_aki/
   [junit4]   2> 1432012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1432012 INFO  (Thread-8885) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1432012 INFO  (Thread-8885) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1432112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkTestServer start zk server on port:39606
   [junit4]   2> 1432112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432116 INFO  (zkCallback-884-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75cc7ce3 name:ZooKeeperConnection Watcher:127.0.0.1:39606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1432119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432121 INFO  (zkCallback-885-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43b84db3 name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1432123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1432124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1432125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1432127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1432127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1432129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1432130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1432131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1432132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1432133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1432133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1432135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1432135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1432136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1432137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1432138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1432138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1432140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1432140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1432141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1432142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1432143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1432143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1432145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1432145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1432420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1
   [junit4]   2> 1432422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1432423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54bcad98{/_aki,null,AVAILABLE}
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2d294442{HTTP/1.1}{127.0.0.1:39301}
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server Started @1435408ms
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores, hostContext=/_aki, hostPort=39301, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/tempDir-001/control/data}
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1432425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/'
   [junit4]   2> 1432425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1432425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1432441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432443 INFO  (zkCallback-886-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a58ee0f name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1432444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/solr.xml
   [junit4]   2> 1432457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores
   [junit4]   2> 1432457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoreContainer New CoreContainer 714441687
   [junit4]   2> 1432457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/]
   [junit4]   2> 1432458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/lib
   [junit4]   2> 1432458 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/lib).
   [junit4]   2> 1432465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39606/solr
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1432468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432469 INFO  (zkCallback-888-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72a2a821 name:ZooKeeperConnection Watcher:127.0.0.1:39606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432472 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e055212 name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1432476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1432478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1432480 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1432482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1432484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1432486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1432487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1432488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1432489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39301__aki
   [junit4]   2> 1432489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39301__aki
   [junit4]   2> 1432491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1432492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1432493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1432494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94815316352565252-127.0.0.1:39301__aki-n_0000000000
   [junit4]   2> 1432494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39301__aki
   [junit4]   2> 1432494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1432495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer Overseer (id=94815316352565252-127.0.0.1:39301__aki-n_0000000000) starting
   [junit4]   2> 1432497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1432502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1432503 INFO  (OverseerCollectionConfigSetProcessor-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1432503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1432503 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1432529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1432529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1432530 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores
   [junit4]   2> 1432531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/, config=solrconfig.xml, collection=control_collection, transient=false, shard=, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, name=collection1, coreNodeName=}
   [junit4]   2> 1432531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/
   [junit4]   2> 1432531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1432532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1432532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1432532 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1432532 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1432534 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1432536 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"} current state version: 0
   [junit4]   2> 1432537 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"}
   [junit4]   2> 1432537 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1432537 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1432539 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/'
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1433551 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1433556 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1433569 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1433623 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1433624 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1433632 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1433879 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1433886 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1433888 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1433898 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1433901 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1433904 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1433905 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1433905 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1433905 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1433907 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1433907 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@82b907
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1433909 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/index
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=59.134765625, floorSegmentMB=1.873046875, forceMergeDeletesPctAllowed=5.101132638817477, segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1433910 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6172c3ac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@358c832f),segFN=segments_1,generation=1}
   [junit4]   2> 1433910 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1433910 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/
   [junit4]   2> 1433911 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1433915 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1433915 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1433915 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1433915 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1433918 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433919 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433920 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433921 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433925 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/admin/file,/admin/properties,/admin/mbeans,/admin/ping,/admin/luke,/admin/plugins,/schema,/replication,/admin/segments,standard,/admin/logging,/dump,/update/json/docs,/update,/admin/threads,/update/json,/config,/get,/admin/system
   [junit4]   2> 1433926 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1433927 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1433927 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5623707623803375]
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6172c3ac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@358c832f),segFN=segments_1,generation=1}
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1433929 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6329c12d[collection1] main
   [junit4]   2> 1433929 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1433931 INFO  (searcherExecutor-3744-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6329c12d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517045063651164160
   [junit4]   2> 1433934 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517045063651164160
   [junit4]   2> 1433934 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1433934 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1433934 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:39301/_aki collection:control_collection shard:shard1
   [junit4]   2> 1433935 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1433940 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94815316352565252-core_node1-n_0000000000
   [junit4]   2> 1433940 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39301/_aki/collection1/
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39301/_aki/collection1/ has no replicas
   [junit4]   2> 1433942 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39301/_aki/collection1/ shard1
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1433945 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1433948 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 1
   [junit4]   2> 1433948 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1433950 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki",
   [junit4]   2>   "core":"collection1"} current state version: 1
   [junit4]   2> 1434034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1434034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1434036 INFO  (zkCallback-891-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f300a00 name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1434036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1434037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1434037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1434039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1434052 INFO  (zkCallback-891-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434052 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:39301/_aki/collection1/ and leader is http://127.0.0.1:39301/_aki/collection1/
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39301/_aki
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1434102 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"} current state version: 2
   [junit4]   2> 1434103 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"}
   [junit4]   2> 1434205 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434205 INFO  (zkCallback-891-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001/cores/collection1
   [junit4]   2> 1434337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001
   [junit4]   2> 1434338 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@769462fe{/_aki,null,AVAILABLE}
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.ServerConnector Started ServerConnector@700bc02d{HTTP/1.1}{127.0.0.1:58896}
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server Started @1437324ms
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostPort=58896, hostContext=/_aki, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/tempDir-001/jetty1}
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001/'
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1434357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1434357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1434359 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21

[...truncated too long message...]

t test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=7AD12F3DB932BF35 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=vi -Dtests.timezone=IET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]    >    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=14042, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   [junit4]    >    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)
   [junit4] Completed [244/545] on J0 in 92.11s, 1 test, 3 errors <<< FAILURES!

[...truncated 921 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2156 tests, 2 suite-level errors, 1 error, 1004 ignored (14 assumptions) [seed: 7AD12F3DB932BF35]

Total time: 165 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1007 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1007/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=27929, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=27929, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45597/v: collection already exists: awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([879120FF3D28603D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
188 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=28412, name=searcherExecutor-3491-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=27631, name=searcherExecutor-2994-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=28976, name=zkCallback-722-thread-24-processing-n:127.0.0.1:44458_v-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)    4) Thread[id=28419, name=searcherExecutor-3483-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=27799, name=searcherExecutor-3120-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=27781, name=searcherExecutor-3124-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=27480, name=Thread-23333, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1155)    8) Thread[id=28851, name=searcherExecutor-3725-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=28974, name=zkCallback-727-thread-5-processing-n:127.0.0.1:42668_v-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   10) Thread[id=27317, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=29043, name=coreZkRegister-2881-thread-24-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=28346, name=searcherExecutor-3410-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=28984, name=coreZkRegister-2881-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=29044, name=coreZkRegister-2881-thread-25-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=27712, name=searcherExecutor-3070-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=27624, name=searcherExecutor-2987-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=28865, name=coreZkRegister-2892-thread-5, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=27460, name=qtp438909521-27460-selector-ServerConnectorManager@135b19c5/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=28374, name=searcherExecutor-3445-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=29025, name=coreZkRegister-2881-thread-11, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=29080, name=zkCallback-722-thread-34, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=28564, name=searcherExecutor-3673-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=28428, name=searcherExecutor-3485-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=28895, name=coreZkRegister-2892-thread-22, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=27801, name=searcherExecutor-3135-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=27763, name=searcherExecutor-3093-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=28263, name=qtp438909521-28263, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=28694, name=zkCallback-727-thread-5, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=28968, name=zkCallback-722-thread-27, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=27630, name=searcherExecutor-2991-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=28841, name=zkCallback-727-thread-13, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=28805, name=zkCallback-722-thread-17, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=28949, name=qtp105842715-28949, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=28893, name=searcherExecutor-3735-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=29078, name=OverseerCollectionConfigSetProcessor-94810750388404291-127.0.0.1:44458_v-n_0000000009, state=TIMED_WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:344)         at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:223)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:197)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=28211, name=qtp438909521-28211, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:881)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:683)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:737)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:703)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:627)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=27798, name=searcherExecutor-3127-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=27770, name=searcherExecutor-3101-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=28523, name=searcherExecutor-3638-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=27468, name=Scheduler-63946838, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=28903, name=searcherExecutor-3742-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=28967, name=zkCallback-722-thread-26-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=29045, name=coreZkRegister-2881-thread-26-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=28997, name=coreZkRegister-2881-thread-4-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=29074, name=zkCallback-722-thread-31-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=29040, name=coreZkRegister-2881-thread-21-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=28492, name=searcherExecutor-3574-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=28460, name=searcherExecutor-3547-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=28868, name=coreZkRegister-2892-thread-8, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=27489, name=Scheduler-853594404, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=28482, name=searcherExecutor-3599-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=27453, name=searcherExecutor-2887-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=29023, name=coreZkRegister-2881-thread-9, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=29029, name=coreZkRegister-2881-thread-15-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=27637, name=searcherExecutor-3010-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=28330, name=searcherExecutor-3377-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=28961, name=zkCallback-727-thread-20, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=28963, name=zkCallback-727-thread-22, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=28964, name=zkCallback-727-thread-23, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=27462, name=qtp438909521-27462-selector-ServerConnectorManager@135b19c5/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=28225, name=qtp438909521-28225, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=28819, name=searcherExecutor-3701-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=27499, name=searcherExecutor-2903-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   64) Thread[id=28379, name=searcherExecutor-3442-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   65) Thread[id=29039, name=coreZkRegister-2881-thread-20-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   66) Thread[id=29038, name=coreZkRegister-2881-thread-19, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=28969, name=zkCallback-722-thread-28, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   68) Thread[id=29076, name=zkCallback-722-thread-33-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=28367, name=searcherExecutor-3433-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   70) Thread[id=27818, name=searcherExecutor-3165-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=28565, name=searcherExecutor-3662-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   72) Thread[id=27320, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)   73) Thread[id=27401, name=qtp1899143395-27401-selector-ServerConnectorManager@71a314d6/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   74) Thread[id=28315, name=qtp1899143395-28315, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   75) Thread[id=28448, name=searcherExecutor-3527-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   76) Thread[id=28502, name=searcherExecutor-3607-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   77) Thread[id=28928, name=qtp105842715-28928, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   78) Thread[id=28975, name=zkCallback-722-thread-24-processing-n:127.0.0.1:44458_v-SendThread(127.0.0.1:37520), state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   79) Thread[id=27896, name=searcherExecutor-3202-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   80) Thread[id=29072, name=zkCallback-722-thread-29-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   81) Thread[id=27425, name=searcherExecutor-2876-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   82) Thread[id=28896, name=coreZkRegister-2892-thread-23, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   83) Thread[id=27587, name=qtp1899143395-27587, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   84) Thread[id=28380, name=searcherExecutor-3449-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   85) Thread[id=29035, name=coreZkRegister-2881-thread-16-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   86) Thread[id=29036, name=coreZkRegister-2881-thread-17-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   87) Thread[id=27647, name=searcherExecutor-3028-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   88) Thread[id=28335, name=searcherExecutor-3383-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   89) Thread[id=27429, name=qtp105842715-27429-selector-ServerConnectorManager@46ba561/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   90) Thread[id=27541, name=searcherExecutor-2924-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at

[...truncated too long message...]

  >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   153) Thread[id=28460, name=searcherExecutor-3547-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   154) Thread[id=28492, name=searcherExecutor-3574-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   155) Thread[id=29040, name=coreZkRegister-2881-thread-21, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   156) Thread[id=29074, name=zkCallback-722-thread-31, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   157) Thread[id=28997, name=coreZkRegister-2881-thread-4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   158) Thread[id=28967, name=zkCallback-722-thread-26, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   159) Thread[id=29045, name=coreZkRegister-2881-thread-26, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   160) Thread[id=28903, name=searcherExecutor-3742-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([879120FF3D28603D]:0)
   [junit4] Completed [425/544] on J0 in 2488.36s, 1 test, 3 errors <<< FAILURES!

[...truncated 349 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151105_040039_516.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid28809.hprof ...
   [junit4] Heap dump file created [643691125 bytes in 6.501 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 20 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 544 suites (7 ignored), 2153 tests, 2 suite-level errors, 1 error, 371 ignored (15 assumptions) [seed: 879120FF3D28603D]

Total time: 183 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1006 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1006/

1 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([F40CBA610A991C8F:735BC7EE9BC0600F]:0)
	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)




Build Log:
[...truncated 8503 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=F40CBA610A991C8F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=pl_PL -Dtests.timezone=WET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    961s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F40CBA610A991C8F:735BC7EE9BC0600F]:0)
   [junit4]    > 	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
   [junit4]    > 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/J1/temp/lucene.search.TestGeoPointQuery_F40CBA610A991C8F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), point=PostingsFormat(name=Direct)}, docValues:{id=DocValuesFormat(name=Asserting), point=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=pl_PL, timezone=WET
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=201486464,total=399507456
   [junit4]   2> NOTE: All tests run in this JVM: [TestGeoUtils, DuplicateFilterTest, TestSlowFuzzyQuery2, TestDocTermOrdsRangeFilter, TestDocValuesNumbersQuery, TestDocValuesRangeQuery, TestDocValuesTermsQuery, TestGeoPointQuery]
   [junit4] Completed [20/20] on J1 in 1050.41s, 22 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/temp/junit4-J1-20151103_222543_473.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid20044.hprof ...
   [junit4] Heap dump file created [544131782 bytes in 4.301 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 20 suites, 141 tests, 1 error, 1 ignored (1 assumption) [seed: F40CBA610A991C8F]

Total time: 117 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1005 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1005/

10 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5614, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5614, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5615, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5615, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5613, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5613, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5610, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5610, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5612, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5612, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 9 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5611, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5611, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 9 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5323, name=OverseerCollectionConfigSetProcessor-94798240480362503-127.0.0.1:43514_-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5323, name=OverseerCollectionConfigSetProcessor-94798240480362503-127.0.0.1:43514_-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: Java heap space


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=64661, name=coreZkRegister-5651-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=64661, name=coreZkRegister-5651-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)




Build Log:
[...truncated 11313 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/init-core-data-001
   [junit4]   2> 3456212 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[30160A4BC582DFA1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /k/ls
   [junit4]   2> 3456231 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3456243 INFO  (Thread-56474) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3456243 INFO  (Thread-56474) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3456343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkTestServer start zk server on port:38299
   [junit4]   2> 3456343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456371 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456383 INFO  (zkCallback-1654-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7bd6f6e1 name:ZooKeeperConnection Watcher:127.0.0.1:38299 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3456385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456391 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456407 INFO  (zkCallback-1655-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@682c7775 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3456409 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3456410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3456411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3456412 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3456412 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3456414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3456414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3456416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3456416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3456417 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3456417 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3456418 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3456419 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3456420 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3456420 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3456421 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3456421 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3456422 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3456423 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3456424 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3456424 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3456425 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3456425 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3456426 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3456427 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3456746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1
   [junit4]   2> 3456748 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3456750 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@706572ee{/k/ls,null,AVAILABLE}
   [junit4]   2> 3456750 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4d47e9c9{HTTP/1.1}{127.0.0.1:47817}
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server Started @3459560ms
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/k/ls, hostPort=47817, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/tempDir-001/control/data}
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/'
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3456767 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456767 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456769 INFO  (zkCallback-1656-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4fab9ec5 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456769 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456769 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456770 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3456770 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/solr.xml
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoreContainer New CoreContainer 134198550
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/]
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/lib
   [junit4]   2> 3456785 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/lib).
   [junit4]   2> 3456793 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3456795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3456795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3456795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3456796 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38299/solr
   [junit4]   2> 3456796 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3456796 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456797 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456799 INFO  (zkCallback-1658-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24f37f1f name:ZooKeeperConnection Watcher:127.0.0.1:38299 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456802 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456803 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456810 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456819 INFO  (zkCallback-1659-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b10c744 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3456821 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3456824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3456825 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3456827 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3456828 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3456829 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3456830 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3456831 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3456832 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47817_k%2Fls
   [junit4]   2> 3456832 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47817_k%2Fls
   [junit4]   2> 3456833 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3456834 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3456835 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3456835 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000
   [junit4]   2> 3456836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47817_k%2Fls
   [junit4]   2> 3456836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3456836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer Overseer (id=94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) starting
   [junit4]   2> 3456838 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3456855 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3456872 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3456872 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3456882 INFO  (OverseerCollectionConfigSetProcessor-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3456895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3456925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3456926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3456927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores
   [junit4]   2> 3456928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, dataDir=data/, schema=schema.xml, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/, name=collection1, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1, configSetProperties=configsetprops.json, collection=control_collection, transient=false, loadOnStartup=true}
   [junit4]   2> 3456929 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/
   [junit4]   2> 3456929 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3456943 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3456943 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3456944 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3456949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3456949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3456950 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 3456950 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 3456950 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3456951 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3456952 INFO  (zkCallback-1659-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3456967 INFO  (zkCallback-1659-thread-2-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3457944 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3457945 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3457945 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3457945 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/'
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3457971 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3457979 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3458004 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3458104 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3458114 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3458121 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3458378 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3458385 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3458387 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3458396 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3458399 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3458402 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3458402 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3458403 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3458403 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3458403 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3458403 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3458404 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3458404 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@51c9a161
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3458406 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/index
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=4, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=1.1650390625, floorSegmentMB=0.4404296875, forceMergeDeletesPctAllowed=4.390988511110557, segmentsPerTier=22.0, maxCFSSegmentSizeMB=1.0302734375, noCFSRatio=1.0
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7e089701 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65f2034b),segFN=segments_1,generation=1}
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3458407 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/
   [junit4]   2> 3458407 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3458412 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3458412 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3458412 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3458415 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458416 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458417 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458419 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458422 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/luke,/admin/threads,/get,/admin/mbeans,/admin/logging,/admin/segments,/replication,/update,/update/csv,standard,/update/json,/update/json/docs,/admin/ping,/admin/file,/admin/system,/admin/plugins,/config,/admin/properties,/schema
   [junit4]   2> 3458423 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3458424 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3458424 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3458424 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.48114508059511085]
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7e089701 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65f2034b),segFN=segments_1,generation=1}
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5665a2ad[collection1] main
   [junit4]   2> 3458426 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3458426 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3458426 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516774576103620608
   [junit4]   2> 3458430 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516774576103620608
   [junit4]   2> 3458430 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3458431 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3458431 INFO  (searcherExecutor-5614-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5665a2ad[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3458435 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:47817/k/ls collection:control_collection shard:shard1
   [junit4]   2> 3458436 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3458439 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94798410864066564-core_node1-n_0000000000
   [junit4]   2> 3458439 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3458440 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3458440 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47817/k/ls/collection1/
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:47817/k/ls/collection1/ has no replicas
   [junit4]   2> 3458441 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47817/k/ls/collection1/ shard1
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3458444 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3458447 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3458451 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3458455 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3458459 INFO  (zkCallback-1661-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f8bbb4 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3458459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3458459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3458459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3458461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3458461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3458461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3458482 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3458485 INFO  (zkCallback-1662-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d9d9433 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3458485 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3458486 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3458488 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 3458488 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3458488 INFO  (zkCallback-1661-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458489 INFO  (zkCallback-1659-thread-2-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458489 INFO  (zkCallback-1659-thread-3-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:47817/k/ls/collection1/ and leader is http://127.0.0.1:47817/k/ls/collection1/
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47817/k/ls
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3458500 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 3458500 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 3458501 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3458604 INFO  (zkCallback-1661-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458604 INFO  (zkCallback-1659-thread-3-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458605 INFO  (zkCallback-1659-thread-2-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/shard-1-001/cores/collection1
   [junit4]   2> 3458911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/shard-1-001
   [junit4]   2> 3458912 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3458967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b2c8c8{/k/ls,null,AVAILABLE}
   [junit4]   2> 3458967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@27dde35e{HTTP/1.1}{127.0.0.1:55839}
   [junit4]   2> 3458967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server Started @3461776ms
  

[...truncated too long message...]

rverException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
   [junit4]    > 	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 moreThrowable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5611, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
   [junit4]    > 	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5323, name=OverseerCollectionConfigSetProcessor-94798240480362503-127.0.0.1:43514_-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4] Completed [495/543] on J1 in 12440.62s, 1 test, 9 errors <<< FAILURES!

[...truncated 142 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151102_230455_660.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid11049.hprof ...
   [junit4] Heap dump file created [664773063 bytes in 21.378 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151102_230455_660.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "TEST-CollectionsAPIDistributedZkTest.test-seed#[30160A4BC582DFA1]-SendThread(127.0.0.1:49324)"
   [junit4] <<< JVM J1: EOF ----

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 8 suite-level errors, 2 errors, 173 ignored (18 assumptions) [seed: 30160A4BC582DFA1]

Total time: 351 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1004 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1004/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15562, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15562, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50716/be_uja/jq: collection already exists: awholynewstresscollection_collection2_2
	at __randomizedtesting.SeedInfo.seed([B8CE9E547BC18E6A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#3): CloudJettyRunner [url=https://127.0.0.1:39313/ynoe/p/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#3): CloudJettyRunner [url=https://127.0.0.1:39313/ynoe/p/collection1]
	at __randomizedtesting.SeedInfo.seed([B8CE9E547BC18E6A:309AA18ED53DE392]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10971 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/init-core-data-001
   [junit4]   2> 1547028 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B8CE9E547BC18E6A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1547028 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B8CE9E547BC18E6A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /be_uja/jq
   [junit4]   2> 1547032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1547033 INFO  (Thread-10104) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1547033 INFO  (Thread-10104) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1547133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkTestServer start zk server on port:39006
   [junit4]   2> 1547133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547138 INFO  (zkCallback-960-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27fc8de3 name:ZooKeeperConnection Watcher:127.0.0.1:39006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1547141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547143 INFO  (zkCallback-961-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62d68c34 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1547146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1547148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1547150 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1547152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1547152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1547155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1547155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1547157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1547158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1547159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1547160 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1547161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1547162 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1547163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1547164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1547166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1547166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1547168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1547168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1547170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1547170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1547172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1547172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1547174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1547174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1547446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1
   [junit4]   2> 1547447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1547449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5852e84a{/be_uja/jq,null,AVAILABLE}
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4827a1fb{HTTP/1.1}{127.0.0.1:48000}
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server Started @1550123ms
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/tempDir-001/control/data, hostContext=/be_uja/jq, hostPort=48000}
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/'
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1547468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547470 INFO  (zkCallback-962-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b6043c7 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1547471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/solr.xml
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1617162446
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/]
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/lib
   [junit4]   2> 1547488 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/lib).
   [junit4]   2> 1547496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1547497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39006/solr
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547500 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@593949a0 name:ZooKeeperConnection Watcher:127.0.0.1:39006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547501 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547501 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547504 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59248b28 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1547507 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1547510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1547512 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1547515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1547517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1547518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1547520 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1547521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1547522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48000_be_uja%2Fjq
   [junit4]   2> 1547522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48000_be_uja%2Fjq
   [junit4]   2> 1547524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1547525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1547527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1547527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000
   [junit4]   2> 1547528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48000_be_uja%2Fjq
   [junit4]   2> 1547528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1547529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer Overseer (id=94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) starting
   [junit4]   2> 1547531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1547538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1547538 INFO  (OverseerCollectionConfigSetProcessor-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1547538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1547539 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1547561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1547561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, collection=control_collection, dataDir=data/, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1, schema=schema.xml, coreNodeName=, shard=, name=collection1, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1547564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1547564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1547565 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1547565 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1547566 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1547567 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1547567 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1547567 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1547568 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1547569 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1548567 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1548567 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1548568 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1548568 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/'
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1548611 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1548617 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1548631 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1548659 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1548660 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1548667 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1548911 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1548918 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1548920 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1548928 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1548931 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1548934 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1548934 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1548935 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1548935 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1548935 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1548935 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1548937 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@459304b2
   [junit4]   2> 1548937 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data
   [junit4]   2> 1548938 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1548938 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1548938 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/index
   [junit4]   2> 1548938 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=27.544921875, floorSegmentMB=0.615234375, forceMergeDeletesPctAllowed=25.515157929851853, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8379542938702577
   [junit4]   2> 1548939 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@dc20826 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3da4fdd8),segFN=segments_1,generation=1}
   [junit4]   2> 1548939 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1548939 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/
   [junit4]   2> 1548939 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1548943 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1548947 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548948 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548950 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548951 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548954 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/update/json/docs,/admin/system,/schema,/update/json,/admin/threads,/admin/file,/admin/mbeans,/admin/logging,/get,/admin/ping,standard,/admin/properties,/admin/segments,/config,/replication,/update/csv,/update,/admin/luke
   [junit4]   2> 1548955 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1548956 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1548956 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@dfcee1d
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@dc20826 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3da4fdd8),segFN=segments_1,generation=1}
   [junit4]   2> 1548958 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1548958 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5723c25f[collection1] main
   [junit4]   2> 1548958 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1548960 INFO  (searcherExecutor-3204-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5723c25f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1548961 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1548961 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1548961 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516685114684211200
   [junit4]   2> 1548963 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516685114684211200
   [junit4]   2> 1548963 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1548963 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1548964 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:48000/be_uja/jq collection:control_collection shard:shard1
   [junit4]   2> 1548964 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1548968 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94792819541475332-core_node1-n_0000000000
   [junit4]   2> 1548968 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48000/be_uja/jq/collection1/
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:48000/be_uja/jq/collection1/ has no replicas
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48000/be_uja/jq/collection1/ shard1
   [junit4]   2> 1548970 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1548973 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1548975 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1549066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549068 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@783c947b name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549072 INFO  (zkCallback-968-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ab1815 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549073 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549075 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150c59130070006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1549075 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1549075 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1549076 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549076 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:48000/be_uja/jq/collection1/ and leader is http://127.0.0.1:48000/be_uja/jq/collection1/
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48000/be_uja/jq
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1549128 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1549128 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1549129 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1549233 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549233 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549344 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1
   [junit4]   2> 1549345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001
   [junit4]   2> 1549345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ec88122{/be_uja/jq,null,AVAILABLE}
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5901bf61{HTTP/1.1}{127.0.0.1:50716}
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server Started @1552018ms
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/tempDir-001/jetty1, hostPort=50716, hostContext=/be_uja/jq, solrconfig=solrconfig.xml}
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/'
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1549363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549365 INFO  (zkCallback-969-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@351a5796 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1549366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/solr.xml
   [junit4]   2> 1549379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores
   [junit4]   2> 1549379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 42163332
   [junit4]   2> 1549379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/]
   [junit4]   2> 1549380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/lib
   [junit4]   2> 1549380 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/lib).
   [junit4]   2> 1549387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39006/solr
   [junit4]   2> 1549389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1549389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549390 INFO  (zkCallback-971-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c9a7170 name:ZooKeeperConnection Watcher:127.0.0.1:39006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549394 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d48ee52 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1550405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50716_be_uja%2Fjq
   [junit4]   2> 1550406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50716_be_uja%2Fjq
   [junit4]   2> 1550407 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1550407 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1550408 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1550410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1550410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94792819541475337-127.0.0.1:50716_be_uja%2Fjq-n_0000000001
   [junit4]   2> 1550411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000 to know if I could be the leader
   [junit4]   2> 1550422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1550422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, transient=false, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1/, shard=, schema=schema.xml, configSetProperties=configsetprops.json, dataDir=data/, collection=collection1, name=collection1}
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1/
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1550425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1550425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1550425 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1550425 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:50716_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:50716/be_uja/jq",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"} current state version: 3
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:50716_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:50716/be_uja/jq",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1550530 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1550531 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1550531 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1551428 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1551428 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1551428 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1551428 INFO  (coreLoadEx

[...truncated too long message...]

/collections/collection1/leader_elect/shard2/election/94792899253764259-core_node6-n_0000000024
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764256-core_node3-n_0000000026
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764214-127.0.0.1:49811_ynoe%2Fp-n_0000000050
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764172-127.0.0.1:58223_ynoe%2Fp-n_0000000029
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764106-127.0.0.1:56562_ynoe%2Fp-n_0000000018
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764136-127.0.0.1:60376_ynoe%2Fp-n_0000000011
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764196-127.0.0.1:56077_ynoe%2Fp-n_0000000044
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764133-core_node10-n_0000000004
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764160-core_node3-n_0000000008
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764115-core_node4-n_0000000001
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764118-core_node5-n_0000000002
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764127-127.0.0.1:41039_ynoe%2Fp-n_0000000008
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764226-core_node8-n_0000000019
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764139-core_node12-n_0000000005
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764283-127.0.0.1:43518_ynoe%2Fp-n_0000000087
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764124-127.0.0.1:56077_ynoe%2Fp-n_0000000007
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764292-127.0.0.1:56077_ynoe%2Fp-n_0000000090
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764220-127.0.0.1:58223_ynoe%2Fp-n_0000000052
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764124-core_node7-n_0000000003
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764178-core_node8-n_0000000012
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	135	/solr/live_nodes
   [junit4]   2> 	69	/solr/collections
   [junit4]   2> 	29	/solr/overseer/collection-queue-work
   [junit4]   2> 	25	/solr/overseer/queue
   [junit4]   2> 	15	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RollingRestartTest -Dtests.method=test -Dtests.seed=B8CE9E547BC18E6A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=vi_VN -Dtests.timezone=GB -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  494s J2 | RollingRestartTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Unable to restart (#3): CloudJettyRunner [url=https://127.0.0.1:39313/ynoe/p/collection1]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B8CE9E547BC18E6A:309AA18ED53DE392]:0)
   [junit4]    > 	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
   [junit4]    > 	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3256946 INFO  (SUITE-RollingRestartTest-seed#[B8CE9E547BC18E6A]-worker) [n:127.0.0.1:38744_ynoe%2Fp c:collection1 s:shard2 r:core_node16 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_B8CE9E547BC18E6A-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=vi_VN, timezone=GB
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=227910440,total=528482304
   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexingPerformance, TermVectorComponentDistributedTest, TestCloudManagedSchemaConcurrent, DirectSolrConnectionTest, TestReRankQParserPlugin, RegexBoostProcessorTest, TestElisionMultitermQuery, TestDefaultSimilarityFactory, SuggesterTSTTest, TestDefaultSearchFieldResource, XmlUpdateRequestHandlerTest, DocValuesMissingTest, TestRebalanceLeaders, CoreMergeIndexesAdminHandlerTest, TestCollationField, TestOrdValues, TestEmbeddedSolrServerConstructors, TestReload, TestFieldTypeCollectionResource, SimpleFacetsTest, LeaderInitiatedRecoveryOnShardRestartTest, SuggesterTest, StatsComponentTest, HLLSerializationTest, DocValuesMultiTest, TestCloudInspectUtil, TestCoreDiscovery, CoreAdminHandlerTest, HdfsWriteToMultipleCollectionsTest, SparseHLLTest, DistribDocExpirationUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, DeleteLastCustomShardedReplicaTest, RulesTest, RuleEngineTest, TestStressRecovery, TestComplexPhraseQParserPlugin, UnloadDistributedZkTest, HttpPartitionTest, TestFuzzyAnalyzedSuggestions, InfoHandlerTest, TestSort, UpdateParamsTest, AliasIntegrationTest, FieldMutatingUpdateProcessorTest, TestSmileRequest, PrimitiveFieldTypeTest, TestPHPSerializedResponseWriter, DistributedQueryComponentCustomSortTest, TestExceedMaxTermLength, TestIndexSearcher, TestSystemIdResolver, TestFastLRUCache, SharedFSAutoReplicaFailoverUtilsTest, SearchHandlerTest, TestScoreJoinQPNoScore, TestJmxIntegration, TestSolrIndexConfig, TestChildDocTransformer, CollectionTooManyReplicasTest, LukeRequestHandlerTest, RecoveryZkTest, CachingDirectoryFactoryTest, SampleTest, TestSweetSpotSimilarityFactory, TestNRTOpen, TestSolrDeletionPolicy1, FullSolrCloudDistribCmdsTest, ChangedSchemaMergeTest, TestDynamicLoading, TestHashQParserPlugin, VMParamsZkACLAndCredentialsProvidersTest, TestConfigSets, TestSchemaVersionResource, TestRequestStatusCollectionAPI, ExplicitHLLTest, TestBulkSchemaAPI, TestSchemaNameResource, SuggesterWFSTTest, TestIntervalFaceting, DistributedFacetPivotLargeTest, TestLeaderElectionZkExpiry, DistributedExpandComponentTest, TestSimpleQParserPlugin, BlockDirectoryTest, TestRecovery, TestXIncludeConfig, QueryParsingTest, BasicDistributedZk2Test, OpenCloseCoreStressTest, AutoCommitTest, TestStressReorder, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, SolrCoreTest, TestGroupingSearch, TestFunctionQuery, DirectUpdateHandlerTest, TestBadConfig, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, SimplePostToolTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, TestStressLucene, SpatialFilterTest, SchemaVersionSpecificBehaviorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, SolrInfoMBeanTest, CacheHeaderTest, TestQueryUtils, TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, BadComponentTest, MultiTermTest, TestSearchPerf, OutputWriterTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, TestLuceneMatchVersion, TestCharFilters, TestCodecSupport, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestFastOutputStream, ScriptEngineTest, URLClassifyProcessorTest, DateMathParserTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestLRUCache, TestUtils, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, CircularListTest, CursorPagingTest, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, ActionThrottleTest, AsyncMigrateRouteKeyTest, CollectionStateFormat2Test, ConnectionManagerTest, DistribCursorPagingTest, DistributedQueueTest, MultiThreadedOCPTest, ReplicationFactorTest, RollingRestartTest]
   [junit4] Completed [442/543] on J2 in 493.75s, 1 test, 1 failure <<< FAILURES!

[...truncated 331 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 1 error, 1 failure, 31 ignored (18 assumptions) [seed: B8CE9E547BC18E6A]

Total time: 286 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



RE: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing

Posted by Martin Gainty <mg...@hotmail.com>.
a few discoveries
almost all of the Abstract Classes and Interfaces for org.apache.lucene.analysis.hunspell.Dictionary have no impl classes for either abstract classes and the interfaces
this causes havoc in the lucene-analyzers-common folder for org.apache.lucene.analysis.hunspell.StemmerTestBase testcaseNotably these Impl Classes were missing in Dictionary:
public class PathImpl implements java.nio.file.Path     
public class ComparatorImpl<T> implements java.util.Comparator<T>
I dont know if these were you the specific issue you came upon but I found ivy's testcase engine lacking
currently re-tasking the more mission critical analyzers to build with maven pom.xml for these reasons:
-easier to unit test-use dependencyManagement, pluginManagement to inherit dependencies and plugins from grandfather pom.xml
-flow follows maven lifecycle instead of bouncing all over the place as ivy bounces from grandchild to grandfather
to sibling...absolutely impossible to follow ivy workflow structure with no lifecycle for the build execution plan to followMavens lifecycle is executed always in the order
Grandfather--> calls to Children --> calls to Grandchildren
each of the grandchildren are executed in specified <module>child1</module> <module>child2</module> order

Are these the failures/errors you are seeing David?
Martin


From: david.w.smiley@gmail.com
Date: Sun, 1 Nov 2015 03:53:24 +0000
Subject: Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing
To: dev@lucene.apache.org

I created an issue: https://issues.apache.org/jira/browse/LUCENE-6873
On Sat, Oct 31, 2015 at 7:52 PM Apache Jenkins Server <je...@builds.apache.org> wrote:
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1003/



1 tests failed.

FAILED:  org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]}



Error Message:

maxY must be >= minY: -23.184599227870876 to -23.18459922787088



Stack Trace:

com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088

        at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)

        at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)

        at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)

        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)

        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)

        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)

        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)

        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)

        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)

        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)

        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)

        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)

        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)

        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)

        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)

        at java.lang.Thread.run(Thread.java:745)









Build Log:

[...truncated 8808 lines...]

   [junit4] Suite: org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 15 cells, 5 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 13 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 21 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 2 cells, 0 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 21 cells, 10 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 5 cells, 5 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 10 cells, 6 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 8 non-zero

   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.

   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom -Dtests.seed=D30126801503C858 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=Asia/Qyzylorda -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

   [junit4] ERROR   0.04s J1 | HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]} <<<

   [junit4]    > Throwable #1: com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088

   [junit4]    >        at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)

   [junit4]    >        at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)

   [junit4]    >        at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)

   [junit4]    >        at java.lang.Thread.run(Thread.java:745)

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 27 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 0 cells, 0 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 10 cells, 9 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 6 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 5 cells, 4 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 11 cells, 9 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 0 cells, 0 non-zero

   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J1/temp/lucene.spatial.prefix.HeatmapFacetCounterTest_D30126801503C858-001

   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=el, timezone=Asia/Qyzylorda

   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=240746280,total=342884352

   [junit4]   2> NOTE: All tests run in this JVM: [TestPointVectorStrategy, TestTestFramework, JtsPolygonTest, HeatmapFacetCounterTest]

   [junit4] Completed [10/23] on J1 in 2.24s, 22 tests, 1 error <<< FAILURES!



[...truncated 50 lines...]

BUILD FAILED

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 23 suites, 235 tests, 1 error, 11 ignored (10 assumptions) [seed: D30126801503C858]



Total time: 201 minutes 15 seconds

Build step 'Invoke Ant' marked build as failure

Archiving artifacts

Recording test results

Email was triggered for: Failure - Any

Sending email for trigger: Failure - Any







---------------------------------------------------------------------

To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org

For additional commands, e-mail: dev-help@lucene.apache.org-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, SpeakerLinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com 		 	   		  

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing

Posted by "david.w.smiley@gmail.com" <da...@gmail.com>.
I created an issue: https://issues.apache.org/jira/browse/LUCENE-6873

On Sat, Oct 31, 2015 at 7:52 PM Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1003/
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom {#11
> seed=[D30126801503C858:4E8BBD0E05B60CEF]}
>
> Error Message:
> maxY must be >= minY: -23.184599227870876 to -23.18459922787088
>
> Stack Trace:
> com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY:
> -23.184599227870876 to -23.18459922787088
>         at
> __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
>         at
> com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
>         at
> org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 8808 lines...]
>    [junit4] Suite: org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 15 cells, 5 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 13 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 21 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 2 cells, 0 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 21 cells, 10 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 5 cells, 5 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 10 cells, 6 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 8 non-zero
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by
> running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom
> -Dtests.seed=D30126801503C858 -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=el -Dtests.timezone=Asia/Qyzylorda -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>    [junit4] ERROR   0.04s J1 | HeatmapFacetCounterTest.testRandom {#11
> seed=[D30126801503C858:4E8BBD0E05B60CEF]} <<<
>    [junit4]    > Throwable #1:
> com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY:
> -23.184599227870876 to -23.18459922787088
>    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
>    [junit4]    >        at
> com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
>    [junit4]    >        at
> org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 27 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 0 cells, 0 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 10 cells, 9 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 6 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 5 cells, 4 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 11 cells, 9 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 0 cells, 0 non-zero
>    [junit4]   2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J1/temp/lucene.spatial.prefix.HeatmapFacetCounterTest_D30126801503C858-001
>    [junit4]   2> NOTE: test params are: codec=Lucene54,
> sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=el,
> timezone=Asia/Qyzylorda
>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
> 1.7.0_72 (64-bit)/cpus=4,threads=1,free=240746280,total=342884352
>    [junit4]   2> NOTE: All tests run in this JVM:
> [TestPointVectorStrategy, TestTestFramework, JtsPolygonTest,
> HeatmapFacetCounterTest]
>    [junit4] Completed [10/23] on J1 in 2.24s, 22 tests, 1 error <<<
> FAILURES!
>
> [...truncated 50 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006:
> There were test failures: 23 suites, 235 tests, 1 error, 11 ignored (10
> assumptions) [seed: D30126801503C858]
>
> Total time: 201 minutes 15 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1003/

1 tests failed.
FAILED:  org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]}

Error Message:
maxY must be >= minY: -23.184599227870876 to -23.18459922787088

Stack Trace:
com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088
	at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
	at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
	at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 8808 lines...]
   [junit4] Suite: org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 15 cells, 5 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 13 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 21 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 2 cells, 0 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 21 cells, 10 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 5 cells, 5 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 10 cells, 6 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 8 non-zero
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom -Dtests.seed=D30126801503C858 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=Asia/Qyzylorda -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.04s J1 | HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]} <<<
   [junit4]    > Throwable #1: com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
   [junit4]    > 	at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
   [junit4]    > 	at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 27 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 0 cells, 0 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 10 cells, 9 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 6 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 5 cells, 4 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 11 cells, 9 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 0 cells, 0 non-zero
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J1/temp/lucene.spatial.prefix.HeatmapFacetCounterTest_D30126801503C858-001
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=el, timezone=Asia/Qyzylorda
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=240746280,total=342884352
   [junit4]   2> NOTE: All tests run in this JVM: [TestPointVectorStrategy, TestTestFramework, JtsPolygonTest, HeatmapFacetCounterTest]
   [junit4] Completed [10/23] on J1 in 2.24s, 22 tests, 1 error <<< FAILURES!

[...truncated 50 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 23 suites, 235 tests, 1 error, 11 ignored (10 assumptions) [seed: D30126801503C858]

Total time: 201 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1002 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1002/

5 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at https://127.0.0.1:36938/xclx/ta: Could not load collection from ZK:acollectionafterbaddelete

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36938/xclx/ta: Could not load collection from ZK:acollectionafterbaddelete
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=8681, name=coreZkRegister-1921-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8681, name=coreZkRegister-1921-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Failed to create backup

Stack Trace:
java.lang.AssertionError: Failed to create backup
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5:622575952FE0CDEA]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.CheckBackupStatus.fetchStatus(CheckBackupStatus.java:51)
	at org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup(TestReplicationHandlerBackup.java:197)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10600 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/init-core-data-001
   [junit4]   2> 739605 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[23AE55F0085E3EA5]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_uv/b
   [junit4]   2> 739610 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 739610 INFO  (Thread-5635) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 739610 INFO  (Thread-5635) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 739710 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkTestServer start zk server on port:48193
   [junit4]   2> 739711 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 739712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 739716 INFO  (zkCallback-691-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791b2bc1 name:ZooKeeperConnection Watcher:127.0.0.1:48193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 739716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 739716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 739716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 739719 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 739720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 739721 INFO  (zkCallback-692-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d493df name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 739721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 739721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 739721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 739724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 739726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 739727 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 739729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 739729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 739731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 739731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 739733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 739734 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 739735 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 739735 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 739737 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 739737 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 739739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 739739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 739740 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 739741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 739742 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 739742 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 739744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 739744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 739746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 739746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 739747 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 739748 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 740030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1
   [junit4]   2> 740032 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 740036 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@18525af5{/_uv/b,null,AVAILABLE}
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6a2422e6{HTTP/1.1}{127.0.0.1:44328}
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server Started @742688ms
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores, hostContext=/_uv/b, hostPort=44328}
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 740039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/'
   [junit4]   2> 740039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 740039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 740055 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 740057 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 740057 INFO  (zkCallback-693-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@fc79d2a name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 740057 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 740058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 740059 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 740059 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/solr.xml
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer New CoreContainer 44516556
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/]
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/lib
   [junit4]   2> 740074 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/lib).
   [junit4]   2> 740083 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48193/solr
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 740085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 740085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 740087 INFO  (zkCallback-695-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e4c277f name:ZooKeeperConnection Watcher:127.0.0.1:48193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 740088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 740088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 740089 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 740091 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b170d2a name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 740091 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 740092 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 740093 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 740107 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 740109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 740111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 740112 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 740114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 740115 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 740116 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 740117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44328__uv%2Fb
   [junit4]   2> 740117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44328__uv%2Fb
   [junit4]   2> 740119 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 740120 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 740121 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 740121 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000
   [junit4]   2> 740122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44328__uv%2Fb
   [junit4]   2> 740122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 740123 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer Overseer (id=94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) starting
   [junit4]   2> 740124 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 740130 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 740130 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 740130 INFO  (OverseerCollectionConfigSetProcessor-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 740131 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 740166 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 740166 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 740167 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores
   [junit4]   2> 740168 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/, config=solrconfig.xml, name=collection1, dataDir=data/, transient=false, collection=control_collection, coreNodeName=, schema=schema.xml, configSetProperties=configsetprops.json, loadOnStartup=true, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1}
   [junit4]   2> 740168 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/
   [junit4]   2> 740168 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 740169 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 740169 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 740169 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 740169 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 740171 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 740172 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 740172 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 740172 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 740172 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 740174 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741172 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 741172 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 741173 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 741173 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/'
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 741189 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 741195 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 741208 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 741236 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 741237 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 741243 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 741485 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 741494 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 741496 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 741509 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 741512 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 741515 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 741516 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 741517 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 741517 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 741517 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70455d96
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/index/
   [junit4]   2> 741519 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/index
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7776191985533545]
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@255d1451 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e0e501),segFN=segments_1,generation=1}
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 741520 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/
   [junit4]   2> 741520 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 741524 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 741526 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 741526 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 741528 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741530 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741531 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741532 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741536 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/schema,/admin/segments,/admin/ping,/admin/properties,/update,/admin/mbeans,/admin/luke,/admin/file,/config,/admin/system,/update/json,/get,/update/json/docs,/admin/plugins,/admin/threads,/admin/logging,standard,/update/csv
   [junit4]   2> 741537 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 741538 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 741538 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.84765625, noCFSRatio=0.30815756924504567]
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@255d1451 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e0e501),segFN=segments_1,generation=1}
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 741540 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@271429c8[collection1] main
   [junit4]   2> 741540 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 741542 INFO  (searcherExecutor-1884-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@271429c8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 741543 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516505190091456512
   [junit4]   2> 741545 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516505190091456512
   [junit4]   2> 741545 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 741545 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 741546 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44328/_uv/b collection:control_collection shard:shard1
   [junit4]   2> 741547 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 741550 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94781574254166020-core_node1-n_0000000000
   [junit4]   2> 741551 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 741552 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 741552 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 741552 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44328/_uv/b/collection1/
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 741553 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44328/_uv/b/collection1/ has no replicas
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44328/_uv/b/collection1/ shard1
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 741556 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 741558 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 741660 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741671 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 741671 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 741673 INFO  (zkCallback-698-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@769927aa name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 741673 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 741673 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 741673 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 741675 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44328/_uv/b/collection1/ and leader is http://127.0.0.1:44328/_uv/b/collection1/
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44328/_uv/b
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 741711 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 741712 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 741814 INFO  (zkCallback-698-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741814 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/cores/collection1
   [junit4]   2> 741950 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001
   [junit4]   2> 741951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 741952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3232c196{/_uv/b,null,AVAILABLE}
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7ba29661{HTTP/1.1}{127.0.0.1:58797}
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server Started @744603ms
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=58797, hostContext=/_uv/b, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/cores}
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/'
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 741970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 741970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 741972 INFO  (zkCallback-699-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@eeef72c name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 741972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 741972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 741973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 741973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/solr.xml
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/cores
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer New CoreContainer 274208664
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/]
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/lib
   [junit4]   2> 741988 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/lib).
   [junit4]   2> 741996 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48193/solr
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 741998 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 741999 INFO  (zkCallback-701-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7fb00281 name:ZooKeeperConnection Watcher:127.0.0.1:48193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 742000 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 742000 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 742002 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:58797__uv%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 742003 INFO  (zkCallback-702-thread-1-processing-n:127.0.0.1:58797__uv%2Fb) [n:127.0.0.1:58797__uv%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@314f2223 name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 742004 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0

[...truncated too long message...]

        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Oct 31, 2015 2:21:58 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2> Oct 31, 2015 2:22:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=Asserting), id=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), text=BlockTreeOrds(blocksize=128), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Lucene54), range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene54)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_DO, timezone=Asia/Hebron
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=73202784,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [TestManagedResourceStorage, TestFieldTypeResource, TestCrossCoreJoin, CoreAdminRequestStatusTest, SpatialRPTFieldTypeTest, TestReloadDeadlock, SegmentsInfoRequestHandlerTest, RequestLoggingTest, TestCopyFieldCollectionResource, TestCoreContainer, TestUtils, TestStressVersions, TestOverriddenPrefixQueryForCustomFieldType, HdfsNNFailoverTest, ChaosMonkeyNothingIsSafeTest, HdfsThreadLeakTest, TestWordDelimiterFilterFactory, SpatialFilterTest, DistributedVersionInfoTest, TermVectorComponentTest, TestObjectReleaseTracker, TestJmxMonitoredMap, TestLFUCache, TestExactStatsCache, WordBreakSolrSpellCheckerTest, SuggestComponentContextFilterQueryTest, DistributedFacetPivotWhiteBoxTest, OverseerTaskQueueTest, SolrInfoMBeanTest, PKIAuthenticationIntegrationTest, PreAnalyzedUpdateProcessorTest, RollingRestartTest, TestFiltering, SpellCheckComponentTest, OverseerStatusTest, BufferStoreTest, TestHighlightDedupGrouping, ClusterStateUpdateTest, TestAddFieldRealTimeGet, SliceStateTest, TestTrieFacet, ChaosMonkeySafeLeaderTest, TestSchemaResource, TestSolrCoreProperties, TestBulkSchemaConcurrent, BasicFunctionalityTest, SaslZkACLProviderTest, SolrXmlInZkTest, SchemaVersionSpecificBehaviorTest, TestHashPartitioner, TestLeaderInitiatedRecoveryThread, SpellPossibilityIteratorTest, AtomicUpdatesTest, MBeansHandlerTest, TestLRUStatsCache, TestInfoStreamLogging, TestNoOpRegenerator, TestRuleBasedAuthorizationPlugin, TestManagedSchemaDynamicFieldResource, CollectionStateFormat2Test, DistanceUnitsTest, RankQueryTest, FastVectorHighlighterTest, DocValuesTest, DistributedFacetPivotSmallTest, CursorMarkTest, ShardRoutingCustomTest, CircularListTest, TestImplicitCoreProperties, TestLMJelinekMercerSimilarityFactory, ShardRoutingTest, DistanceFunctionTest, IndexBasedSpellCheckerTest, TestAnalyzeInfixSuggestions, SpellCheckCollatorTest, TestSuggestSpellingConverter, TestHdfsUpdateLog, LoggingHandlerTest, TestRTGBase, TestBlendedInfixSuggestions, SuggesterFSTTest, SecurityConfHandlerTest, SolrIndexSplitterTest, TermsComponentTest, TestBM25SimilarityFactory, NotRequiredUniqueKeyTest, ZkCLITest, TestSortingResponseWriter, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=23AE55F0085E3EA5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_DO -Dtests.timezone=Asia/Hebron -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)
   [junit4] Completed [543/543] on J1 in 4836.62s, 1 test, 3 errors <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151030_202826_187.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18125.hprof ...
   [junit4] Heap dump file created [630003199 bytes in 11.073 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 2 suite-level errors, 2 errors, 1 failure, 31 ignored (18 assumptions) [seed: 23AE55F0085E3EA5]

Total time: 246 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1001 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1001/

4 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
KeeperErrorCode = Session expired for /clusterstate.json

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:477)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:257)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForCollectionToDisappear(AbstractDistribZkTestBase.java:196)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.assertCollectionNotExists(AbstractFullDistribZkTestBase.java:1772)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection(CollectionsAPIDistributedZkTest.java:234)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:171)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=606, name=coreZkRegister-205-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=606, name=coreZkRegister-205-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10206 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/init-core-data-001
   [junit4]   2> 111819 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[6263BEB3B251C403]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ktdm/
   [junit4]   2> 111824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 111824 INFO  (Thread-155) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 111824 INFO  (Thread-155) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 111924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkTestServer start zk server on port:33745
   [junit4]   2> 111924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 111925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 111928 INFO  (zkCallback-55-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3959b9e2 name:ZooKeeperConnection Watcher:127.0.0.1:33745 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 111928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 111928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 111928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 111931 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 111932 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 111934 INFO  (zkCallback-56-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b615046 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 111934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 111934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 111934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 111937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 111940 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 111943 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 111946 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 111946 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 111951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 111951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 111954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 111954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 111957 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 111958 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 111962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 111962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 111965 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 111965 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 111968 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 111969 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 111972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 111972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 111975 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 111975 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 111979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 111980 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 111984 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 111985 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 112389 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1
   [junit4]   2> 112391 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 112396 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f5fd0{/ktdm,null,AVAILABLE}
   [junit4]   2> 112407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6b8f13f8{HTTP/1.1}{127.0.0.1:40092}
   [junit4]   2> 112413 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server Started @115143ms
   [junit4]   2> 112413 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/tempDir-001/control/data, hostPort=40092, hostContext=/ktdm, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores}
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/'
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 112432 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 112434 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 112437 INFO  (zkCallback-57-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18d14949 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 112437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 112438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 112441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 112441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/solr.xml
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer New CoreContainer 1856592448
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/]
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/lib
   [junit4]   2> 112463 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/lib).
   [junit4]   2> 112472 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33745/solr
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 112474 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 112475 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 112478 INFO  (zkCallback-59-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34980b87 name:ZooKeeperConnection Watcher:127.0.0.1:33745 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 112479 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 112480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 112486 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 112489 INFO  (zkCallback-60-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c9c1bb2 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 112489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 112493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 112498 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 112503 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 112511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 112517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 112524 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 112528 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 112530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 112532 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 112537 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40092_ktdm
   [junit4]   2> 112537 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40092_ktdm
   [junit4]   2> 112541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 112544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 112548 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 112549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94775665174380548-127.0.0.1:40092_ktdm-n_0000000000
   [junit4]   2> 112550 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40092_ktdm
   [junit4]   2> 112550 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 112552 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer Overseer (id=94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) starting
   [junit4]   2> 112556 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 112570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 112571 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 112571 INFO  (OverseerCollectionConfigSetProcessor-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 112572 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 112583 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 112610 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 112610 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 112611 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores
   [junit4]   2> 112612 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, coreNodeName=, config=solrconfig.xml, transient=false, collection=control_collection, schema=schema.xml, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1, name=collection1}
   [junit4]   2> 112612 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/
   [junit4]   2> 112612 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 112613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 112613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 112614 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 112614 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 112621 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 112625 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 112632 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 112633 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 112634 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 112646 INFO  (zkCallback-60-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 112653 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 113634 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 113634 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 113635 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 113635 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/'
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 113654 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 113672 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 113707 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 113742 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 113744 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 113787 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 114168 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 114180 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 114182 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 114220 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 114238 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 114241 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 114242 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 114242 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 114242 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 114243 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 114244 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36036784
   [junit4]   2> 114247 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data
   [junit4]   2> 114247 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/index/
   [junit4]   2> 114247 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 114247 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/index
   [junit4]   2> 114248 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12810220323990962]
   [junit4]   2> 114248 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@63195110 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13192444),segFN=segments_1,generation=1}
   [junit4]   2> 114248 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 114266 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/
   [junit4]   2> 114266 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 114295 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 114310 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 114310 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 114310 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 114313 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114314 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114316 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114318 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114329 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /config,/admin/ping,/admin/threads,/update/json/docs,standard,/admin/plugins,/admin/system,/admin/mbeans,/get,/admin/file,/schema,/update/json,/admin/segments,/update,/admin/luke,/admin/logging,/replication,/admin/properties,/update/csv
   [junit4]   2> 114333 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 114334 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 114334 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 114335 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 114335 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 114335 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@29955a9c
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@63195110 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13192444),segFN=segments_1,generation=1}
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4e1eabcc[collection1] main
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 114338 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 114341 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 114341 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 114341 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516410645425160192
   [junit4]   2> 114343 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516410645425160192
   [junit4]   2> 114343 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 114344 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 114346 INFO  (searcherExecutor-168-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4e1eabcc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 114354 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:40092/ktdm collection:control_collection shard:shard1
   [junit4]   2> 114356 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 114370 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94775665174380548-core_node1-n_0000000000
   [junit4]   2> 114371 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40092/ktdm/collection1/
   [junit4]   2> 114377 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40092/ktdm/collection1/ has no replicas
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40092/ktdm/collection1/ shard1
   [junit4]   2> 114378 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 114378 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114379 INFO  (zkCallback-60-thread-3-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114388 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 114394 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 114499 INFO  (zkCallback-60-thread-3-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114499 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:40092/ktdm/collection1/ and leader is http://127.0.0.1:40092/ktdm/collection1/
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40092/ktdm
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 114547 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 114549 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 114550 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 114615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 114617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 114619 INFO  (zkCallback-62-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1cc85d01 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 114619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 114620 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 114620 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 114624 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 114653 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114653 INFO  (zkCallback-62-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114654 INFO  (zkCallback-60-thread-3-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114931 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/cores/collection1
   [junit4]   2> 114931 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001
   [junit4]   2> 114933 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2fc16b7{/ktdm,null,AVAILABLE}
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.ServerConnector Started ServerConnector@743749e8{HTTP/1.1}{127.0.0.1:43568}
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server Started @117667ms
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/cores, hostContext=/ktdm, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostPort=43568}
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/'
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 114956 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 114957 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 114959 INFO  (zkCallback-63-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d14d58b name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 114959 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 114960 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 114961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 114961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/solr.xml
   [junit4]   2> 114978 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/cores
   [junit4]   2> 114979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer New CoreContainer 1964988894
   [junit4]   2> 114979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/]
   [junit4]   2> 114979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/lib
   [junit4]   2> 114979 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/lib).
   [junit4]   2> 115005 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 115006 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 115006 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33745/solr
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 115009 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 115010 INFO  (zkCallback-65-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5a706271 name:ZooKeeperConnection Watcher:127.0.0.1:33745 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 115011 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 115012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 115018 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:43568_ktdm    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 115021 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:43568_ktdm) [n:127.0.0.1:43568_ktdm    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e1eb68f name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 115021 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:43568_ktdm    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 115030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.

[...truncated too long message...]

    1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>    2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2> Oct 29, 2015 8:16:05 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_GT, timezone=Asia/Istanbul
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=3,free=124637208,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [TestPartialUpdateDeduplication, SolrInfoMBeanTest, BlockCacheTest, SegmentsInfoRequestHandlerTest, BufferStoreTest, TestSolrDynamicMBean, TestExactStatsCache, SimpleMLTQParserTest, TestBM25SimilarityFactory, NotRequiredUniqueKeyTest, TestLeaderInitiatedRecoveryThread, DocumentBuilderTest, OpenCloseCoreStressTest, ZkCLITest, TestImplicitCoreProperties, SortByFunctionTest, SuggesterFSTTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DistributedFacetPivotSmallTest, HdfsBasicDistributedZkTest, UpdateRequestProcessorFactoryTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestAnalyzeInfixSuggestions, CollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=6263BEB3B251C403 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_GT -Dtests.timezone=Asia/Istanbul -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)
   [junit4] Completed [216/543] on J1 in 711.72s, 1 test, 3 errors <<< FAILURES!

[...truncated 972 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151029_193608_466.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid26869.hprof ...
   [junit4] Heap dump file created [605716282 bytes in 5.090 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 18 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 2 suite-level errors, 2 errors, 1139 ignored (15 assumptions) [seed: 6263BEB3B251C403]

Total time: 166 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1000 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1000/

1 tests failed.
FAILED:  org.apache.solr.cloud.TestReplicaProperties.test

Error Message:
Collection testcollection does not have roles evenly distributed. Collection is: DocCollection(testcollection)={   "shards":{     "shard1":{       "range":"80000000-b332ffff",       "state":"active",       "replicas":{         "core_node7":{           "base_url":"http://127.0.0.1:51380",           "node_name":"127.0.0.1:51380_",           "core":"testcollection_shard1_replica2",           "state":"active",           "leader":"true"},         "core_node9":{           "base_url":"http://127.0.0.1:45841",           "node_name":"127.0.0.1:45841_",           "core":"testcollection_shard1_replica1",           "state":"active",           "property.preferredleader":"true"},         "core_node10":{           "base_url":"http://127.0.0.1:57326",           "node_name":"127.0.0.1:57326_",           "core":"testcollection_shard1_replica3",           "state":"active"}}},     "shard2":{       "range":"b3330000-e665ffff",       "state":"active",       "replicas":{         "core_node1":{           "base_url":"http://127.0.0.1:42012",           "node_name":"127.0.0.1:42012_",           "core":"testcollection_shard2_replica1",           "state":"active"},         "core_node8":{           "base_url":"http://127.0.0.1:45841",           "node_name":"127.0.0.1:45841_",           "core":"testcollection_shard2_replica3",           "state":"active",           "leader":"true"},         "core_node15":{           "base_url":"http://127.0.0.1:34598",           "node_name":"127.0.0.1:34598_",           "core":"testcollection_shard2_replica2",           "state":"active",           "property.preferredleader":"true"}}},     "shard3":{       "range":"e6660000-1998ffff",       "state":"active",       "replicas":{         "core_node2":{           "base_url":"http://127.0.0.1:57326",           "node_name":"127.0.0.1:57326_",           "core":"testcollection_shard3_replica2",           "state":"active"},         "core_node5":{           "base_url":"http://127.0.0.1:51380",           "node_name":"127.0.0.1:51380_",           "core":"testcollection_shard3_replica1",           "state":"active",           "leader":"true",           "property.preferredleader":"true"},         "core_node14":{           "base_url":"http://127.0.0.1:42012",           "node_name":"127.0.0.1:42012_",           "core":"testcollection_shard3_replica3",           "state":"active"}}},     "shard4":{       "range":"19990000-4ccbffff",       "state":"active",       "replicas":{         "core_node4":{           "base_url":"http://127.0.0.1:51380",           "node_name":"127.0.0.1:51380_",           "core":"testcollection_shard4_replica3",           "state":"active",           "leader":"true"},         "core_node11":{           "base_url":"http://127.0.0.1:45841",           "node_name":"127.0.0.1:45841_",           "core":"testcollection_shard4_replica2",           "state":"active",           "property.preferredleader":"true"},         "core_node12":{           "base_url":"http://127.0.0.1:34598",           "node_name":"127.0.0.1:34598_",           "core":"testcollection_shard4_replica1",           "state":"active"}}},     "shard5":{       "range":"4ccc0000-7fffffff",       "state":"active",       "replicas":{         "core_node3":{           "base_url":"http://127.0.0.1:42012",           "node_name":"127.0.0.1:42012_",           "core":"testcollection_shard5_replica2",           "state":"active",           "leader":"true"},         "core_node6":{           "base_url":"http://127.0.0.1:57326",           "node_name":"127.0.0.1:57326_",           "core":"testcollection_shard5_replica1",           "state":"active",           "property.preferredleader":"true"},         "core_node13":{           "base_url":"http://127.0.0.1:34598",           "node_name":"127.0.0.1:34598_",           "core":"testcollection_shard5_replica3",           "state":"active"}}}},   "replicationFactor":"3",   "autoAddReplicas":"false",   "maxShardsPerNode":"16",   "router":{"name":"compositeId"}}

Stack Trace:
java.lang.AssertionError: Collection testcollection does not have roles evenly distributed. Collection is: DocCollection(testcollection)={
  "shards":{
    "shard1":{
      "range":"80000000-b332ffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "base_url":"http://127.0.0.1:51380",
          "node_name":"127.0.0.1:51380_",
          "core":"testcollection_shard1_replica2",
          "state":"active",
          "leader":"true"},
        "core_node9":{
          "base_url":"http://127.0.0.1:45841",
          "node_name":"127.0.0.1:45841_",
          "core":"testcollection_shard1_replica1",
          "state":"active",
          "property.preferredleader":"true"},
        "core_node10":{
          "base_url":"http://127.0.0.1:57326",
          "node_name":"127.0.0.1:57326_",
          "core":"testcollection_shard1_replica3",
          "state":"active"}}},
    "shard2":{
      "range":"b3330000-e665ffff",
      "state":"active",
      "replicas":{
        "core_node1":{
          "base_url":"http://127.0.0.1:42012",
          "node_name":"127.0.0.1:42012_",
          "core":"testcollection_shard2_replica1",
          "state":"active"},
        "core_node8":{
          "base_url":"http://127.0.0.1:45841",
          "node_name":"127.0.0.1:45841_",
          "core":"testcollection_shard2_replica3",
          "state":"active",
          "leader":"true"},
        "core_node15":{
          "base_url":"http://127.0.0.1:34598",
          "node_name":"127.0.0.1:34598_",
          "core":"testcollection_shard2_replica2",
          "state":"active",
          "property.preferredleader":"true"}}},
    "shard3":{
      "range":"e6660000-1998ffff",
      "state":"active",
      "replicas":{
        "core_node2":{
          "base_url":"http://127.0.0.1:57326",
          "node_name":"127.0.0.1:57326_",
          "core":"testcollection_shard3_replica2",
          "state":"active"},
        "core_node5":{
          "base_url":"http://127.0.0.1:51380",
          "node_name":"127.0.0.1:51380_",
          "core":"testcollection_shard3_replica1",
          "state":"active",
          "leader":"true",
          "property.preferredleader":"true"},
        "core_node14":{
          "base_url":"http://127.0.0.1:42012",
          "node_name":"127.0.0.1:42012_",
          "core":"testcollection_shard3_replica3",
          "state":"active"}}},
    "shard4":{
      "range":"19990000-4ccbffff",
      "state":"active",
      "replicas":{
        "core_node4":{
          "base_url":"http://127.0.0.1:51380",
          "node_name":"127.0.0.1:51380_",
          "core":"testcollection_shard4_replica3",
          "state":"active",
          "leader":"true"},
        "core_node11":{
          "base_url":"http://127.0.0.1:45841",
          "node_name":"127.0.0.1:45841_",
          "core":"testcollection_shard4_replica2",
          "state":"active",
          "property.preferredleader":"true"},
        "core_node12":{
          "base_url":"http://127.0.0.1:34598",
          "node_name":"127.0.0.1:34598_",
          "core":"testcollection_shard4_replica1",
          "state":"active"}}},
    "shard5":{
      "range":"4ccc0000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "base_url":"http://127.0.0.1:42012",
          "node_name":"127.0.0.1:42012_",
          "core":"testcollection_shard5_replica2",
          "state":"active",
          "leader":"true"},
        "core_node6":{
          "base_url":"http://127.0.0.1:57326",
          "node_name":"127.0.0.1:57326_",
          "core":"testcollection_shard5_replica1",
          "state":"active",
          "property.preferredleader":"true"},
        "core_node13":{
          "base_url":"http://127.0.0.1:34598",
          "node_name":"127.0.0.1:34598_",
          "core":"testcollection_shard5_replica3",
          "state":"active"}}}},
  "replicationFactor":"3",
  "autoAddReplicas":"false",
  "maxShardsPerNode":"16",
  "router":{"name":"compositeId"}}
	at __randomizedtesting.SeedInfo.seed([D08670B1FAA54301:58D24F6B54592EF9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUnique(ReplicaPropertiesBase.java:176)
	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUniqueAcrossCollection(ReplicaPropertiesBase.java:107)
	at org.apache.solr.cloud.TestReplicaProperties.clusterAssignPropertyTest(TestReplicaProperties.java:108)
	at org.apache.solr.cloud.TestReplicaProperties.test(TestReplicaProperties.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11383 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestReplicaProperties
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/init-core-data-001
   [junit4]   2> 3277076 WARN  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=22 numCloses=22
   [junit4]   2> 3277076 INFO  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 3277076 INFO  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 3277081 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3277081 INFO  (Thread-6689) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3277081 INFO  (Thread-6689) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3277181 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkTestServer start zk server on port:47499
   [junit4]   2> 3277181 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277182 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277185 INFO  (zkCallback-2471-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1bdb49b name:ZooKeeperConnection Watcher:127.0.0.1:47499 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277185 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277185 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277185 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3277188 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277188 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277189 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f757b19 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277189 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277190 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277190 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3277192 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3277193 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3277195 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3277196 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3277196 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3277198 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 3277199 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3277200 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3277201 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3277202 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3277202 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3277204 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3277204 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3277205 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3277205 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3277207 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3277207 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3277209 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3277209 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3277210 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3277210 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3277212 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3277212 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3277213 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3277214 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3277501 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1
   [junit4]   2> 3277502 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1157e1a5{/,null,AVAILABLE}
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2f10d173{HTTP/1.1}{127.0.0.1:57326}
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server Started @3280459ms
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores, hostPort=57326}
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 3277505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/'
   [junit4]   2> 3277505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3277505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3277521 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277521 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277523 INFO  (zkCallback-2473-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@450f73e5 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277523 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277523 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277524 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3277524 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/solr.xml
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer New CoreContainer 1578509904
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/]
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/lib
   [junit4]   2> 3277538 WARN  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/lib).
   [junit4]   2> 3277546 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3277547 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47499/solr
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277549 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277550 INFO  (zkCallback-2475-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f65dd9f name:ZooKeeperConnection Watcher:127.0.0.1:47499 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277550 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277551 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277552 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277553 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17ea5ef5 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277553 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277554 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3277556 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3277559 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3277560 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3277562 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3277565 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3277566 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3277568 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3277569 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3277570 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57326_
   [junit4]   2> 3277570 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57326_
   [junit4]   2> 3277572 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3277573 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3277574 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3277575 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94771138685435908-127.0.0.1:57326_-n_0000000000
   [junit4]   2> 3277575 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57326_
   [junit4]   2> 3277575 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3277576 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer Overseer (id=94771138685435908-127.0.0.1:57326_-n_0000000000) starting
   [junit4]   2> 3277578 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3277584 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3277584 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3277584 INFO  (OverseerCollectionConfigSetProcessor-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3277584 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3277609 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3277609 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3277610 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores
   [junit4]   2> 3277611 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, loadOnStartup=true, collection=control_collection, name=collection1, shard=, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1, coreNodeName=, dataDir=data/, transient=false, schema=schema.xml}
   [junit4]   2> 3277611 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/
   [junit4]   2> 3277611 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3277612 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3277612 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3277612 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3277612 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3277614 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 3277614 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3277615 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3277615 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3277615 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3277617 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3278615 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3278615 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3278616 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3278616 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/'
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3278632 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3278638 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3278651 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3278680 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3278681 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3278687 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3278904 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3278906 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3278907 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3278910 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e1a3715
   [junit4]   2> 3278930 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data
   [junit4]   2> 3278930 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3278930 WARN  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3278930 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/index
   [junit4]   2> 3278931 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3183713194007609]
   [junit4]   2> 3278934 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6781a840),segFN=segments_1,generation=1}
   [junit4]   2> 3278934 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3278934 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/
   [junit4]   2> 3278935 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 3278938 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3278938 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3278942 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278943 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278945 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278946 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278949 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/plugins,/update,/admin/mbeans,/admin/segments,/schema,/admin/ping,/admin/logging,/admin/file,/update/json,/get,/update/json/docs,/update/csv,/admin/properties,/replication,/config,/admin/system,/admin/threads,/admin/luke
   [junit4]   2> 3278951 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3278951 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3278951 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3278952 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3278952 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3278952 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3278953 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6781a840),segFN=segments_1,generation=1}
   [junit4]   2> 3278953 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3278954 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2ba9a561[collection1] main
   [junit4]   2> 3278954 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3278956 INFO  (searcherExecutor-6371-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2ba9a561[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516338220933054464
   [junit4]   2> 3278959 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516338220933054464
   [junit4]   2> 3278959 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3278959 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3278960 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57326 collection:control_collection shard:shard1
   [junit4]   2> 3278961 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3278964 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94771138685435908-core_node1-n_0000000000
   [junit4]   2> 3278965 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3278966 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57326/collection1/
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57326/collection1/ has no replicas
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57326/collection1/ shard1
   [junit4]   2> 3278967 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3278970 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3278972 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3279075 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3279114 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279114 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279116 INFO  (zkCallback-2478-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73be9c name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279116 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279116 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279116 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3279119 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3279119 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3279119 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279122 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279123 INFO  (zkCallback-2479-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59848a94 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279123 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57326/collection1/ and leader is http://127.0.0.1:57326/collection1/
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57326
   [junit4]   2> 3279123 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3279126 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 3279127 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3279128 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 3279129 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3279131 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3279131 INFO  (zkCallback-2478-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3279232 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3279426 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1
   [junit4]   2> 3279427 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001
   [junit4]   2> 3279428 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3279431 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d431817{/,null,AVAILABLE}
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5fe2bdd6{HTTP/1.1}{127.0.0.1:51380}
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server Started @3282387ms
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/tempDir-001/jetty1, hostPort=51380, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores, hostContext=/}
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/'
   [junit4]   2> 3279433 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3279433 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3279448 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279449 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279450 INFO  (zkCallback-2480-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@858e8fb name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279451 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279451 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279452 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3279452 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/solr.xml
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer New CoreContainer 805970313
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/]
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/lib
   [junit4]   2> 3279465 WARN  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/lib).
   [junit4]   2> 3279473 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3279474 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47499/solr
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279476 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279477 INFO  (zkCallback-2482-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6923f55f name:ZooKeeperConnection Watcher:127.0.0.1:47499 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279477 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279477 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279479 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279480 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57cc1883 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279480 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279485 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3280490 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51380_
   [junit4]   2> 3280491 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51380_
   [junit4]   2> 3280492 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3280492 INFO  (zkCallback-2478-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3280492 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3280494 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3280495 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94771138685435913-127.0.0.1:51380_-n_0000000001
   [junit4]   2> 3280496 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94771138685435908-127.0.0.1:57326_-n_0000000000 to know if I could be the leader
   [junit4]   2> 3280505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3280505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3280506 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores
   [junit4]   2> 3280507 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, coreNodeName=, shard=, transient=false, config=solrconfig.xml, schema=schema.xml, collection=collection1, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1, loadOnStartup=true, configSetProperties=configsetprops.json}
   [junit4]   2> 3280507 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1/
   [junit4]   2> 3280507 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3280508 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3280508 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3280508 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3280508 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3280509 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 3280510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51380",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51380_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 3280510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 3280510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51380",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51380_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 3280613 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 3280613 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3280614 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 3281510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3281510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 3281511 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3281511 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1/'
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3281527 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.

[...truncated too long message...]

 "core":"testcollection_shard3_replica3",
   [junit4]    >           "state":"active"}}},
   [junit4]    >     "shard4":{
   [junit4]    >       "range":"19990000-4ccbffff",
   [junit4]    >       "state":"active",
   [junit4]    >       "replicas":{
   [junit4]    >         "core_node4":{
   [junit4]    >           "base_url":"http://127.0.0.1:51380",
   [junit4]    >           "node_name":"127.0.0.1:51380_",
   [junit4]    >           "core":"testcollection_shard4_replica3",
   [junit4]    >           "state":"active",
   [junit4]    >           "leader":"true"},
   [junit4]    >         "core_node11":{
   [junit4]    >           "base_url":"http://127.0.0.1:45841",
   [junit4]    >           "node_name":"127.0.0.1:45841_",
   [junit4]    >           "core":"testcollection_shard4_replica2",
   [junit4]    >           "state":"active",
   [junit4]    >           "property.preferredleader":"true"},
   [junit4]    >         "core_node12":{
   [junit4]    >           "base_url":"http://127.0.0.1:34598",
   [junit4]    >           "node_name":"127.0.0.1:34598_",
   [junit4]    >           "core":"testcollection_shard4_replica1",
   [junit4]    >           "state":"active"}}},
   [junit4]    >     "shard5":{
   [junit4]    >       "range":"4ccc0000-7fffffff",
   [junit4]    >       "state":"active",
   [junit4]    >       "replicas":{
   [junit4]    >         "core_node3":{
   [junit4]    >           "base_url":"http://127.0.0.1:42012",
   [junit4]    >           "node_name":"127.0.0.1:42012_",
   [junit4]    >           "core":"testcollection_shard5_replica2",
   [junit4]    >           "state":"active",
   [junit4]    >           "leader":"true"},
   [junit4]    >         "core_node6":{
   [junit4]    >           "base_url":"http://127.0.0.1:57326",
   [junit4]    >           "node_name":"127.0.0.1:57326_",
   [junit4]    >           "core":"testcollection_shard5_replica1",
   [junit4]    >           "state":"active",
   [junit4]    >           "property.preferredleader":"true"},
   [junit4]    >         "core_node13":{
   [junit4]    >           "base_url":"http://127.0.0.1:34598",
   [junit4]    >           "node_name":"127.0.0.1:34598_",
   [junit4]    >           "core":"testcollection_shard5_replica3",
   [junit4]    >           "state":"active"}}}},
   [junit4]    >   "replicationFactor":"3",
   [junit4]    >   "autoAddReplicas":"false",
   [junit4]    >   "maxShardsPerNode":"16",
   [junit4]    >   "router":{"name":"compositeId"}}
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D08670B1FAA54301:58D24F6B54592EF9]:0)
   [junit4]    > 	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUnique(ReplicaPropertiesBase.java:176)
   [junit4]    > 	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUniqueAcrossCollection(ReplicaPropertiesBase.java:107)
   [junit4]    > 	at org.apache.solr.cloud.TestReplicaProperties.clusterAssignPropertyTest(TestReplicaProperties.java:108)
   [junit4]    > 	at org.apache.solr.cloud.TestReplicaProperties.test(TestReplicaProperties.java:70)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3341181 INFO  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [n:127.0.0.1:42012_ c:testcollection s:shard3 r:core_node14 x:testcollection_shard3_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001
   [junit4]   2> Oct 29, 2015 12:27:58 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ar_DZ, timezone=America/North_Dakota/Beulah
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=116519584,total=524288000
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldTypeCollectionResource, TestSystemIdResolver, TestExactSharedStatsCache, TestCryptoKeys, AliasIntegrationTest, ShowFileRequestHandlerTest, TestNRTOpen, QueryParsingTest, TestReRankQParserPlugin, TestChildDocTransformer, LeaderInitiatedRecoveryOnShardRestartTest, AutoCommitTest, HLLSerializationTest, TestDefaultSearchFieldResource, MergeStrategyTest, TestHashQParserPlugin, TestZkChroot, XmlUpdateRequestHandlerTest, TestSimpleQParserPlugin, DirectUpdateHandlerOptimizeTest, InfoHandlerTest, SuggesterTSTTest, MultiThreadedOCPTest, SharedFSAutoReplicaFailoverTest, CurrencyFieldXmlFileTest, TestCSVResponseWriter, TestSmileRequest, TestCustomDocTransformer, TestConfigSetsAPI, FieldAnalysisRequestHandlerTest, BinaryUpdateRequestHandlerTest, RulesTest, UUIDFieldTest, TestDynamicFieldCollectionResource, BadIndexSchemaTest, LukeRequestHandlerTest, TestSolrQueryParser, BadComponentTest, TestPostingsSolrHighlighter, DistributedSuggestComponentTest, DistributedQueryComponentCustomSortTest, BlockDirectoryTest, TestCharFilters, SuggesterWFSTTest, TestCloudInspectUtil, TestPHPSerializedResponseWriter, TestSolrIndexConfig, TestComplexPhraseQParserPlugin, HighlighterConfigTest, TestFastWriter, UpdateParamsTest, PluginInfoTest, HttpPartitionTest, HardAutoCommitTest, OverseerRolesTest, SuggesterTest, SampleTest, DocValuesMissingTest, DirectSolrConnectionTest, TestLRUCache, TestJsonRequest, TestCollationField, TestCloudManagedSchemaConcurrent, TestElisionMultitermQuery, QueryResultKeyTest, TestReversedWildcardFilterFactory, TestSolrDeletionPolicy1, TestSolrQueryParserResource, TestCloudPivotFacet, TestFileDictionaryLookup, StatelessScriptUpdateProcessorFactoryTest, DeleteLastCustomShardedReplicaTest, TestGroupingSearch, PrimitiveFieldTypeTest, CoreAdminHandlerTest, SolrCoreCheckLockOnStartupTest, TestStressReorder, TestScoreJoinQPScore, DocValuesMultiTest, TestConfigSetImmutable, TestFastLRUCache, RuleEngineTest, AnalyticsMergeStrategyTest, HdfsWriteToMultipleCollectionsTest, TestSort, SharedFSAutoReplicaFailoverUtilsTest, OpenCloseCoreStressTest, LeaderElectionTest, ZkSolrClientTest, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestRealTimeGet, TestJoin, TestReload, TestRangeQuery, SimpleFacetsTest, TestSolr4Spatial, StatsComponentTest, SpellCheckComponentTest, PeerSyncTest, TestFiltering, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, TestExtendedDismaxParser, SpellCheckCollatorTest, SolrRequestParserTest, SpatialFilterTest, PolyFieldTest, NoCacheHeaderTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, DefaultValueUpdateProcessorTest, SortByFunctionTest, TestRemoteStreaming, SolrInfoMBeanTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestWriterPerf, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TermsComponentTest, RequestHandlersTest, TestValueSourceCache, TermVectorComponentTest, TestIndexingPerformance, TestArbitraryIndexDir, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, SolrPluginUtilsTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestPartialUpdateDeduplication, TestComponentsName, SearchHandlerTest, TestLFUCache, TestQuerySenderListener, AlternateDirectoryTest, TestQuerySenderNoQuery, ResponseLogComponentTest, TestStressRecovery, NotRequiredUniqueKeyTest, TestPhraseSuggestions, TestXIncludeConfig, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, ResourceLoaderTest, URLClassifyProcessorTest, ChaosMonkeyNothingIsSafeTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestEmbeddedSolrServerConstructors, ActionThrottleTest, CollectionReloadTest, ConcurrentDeleteAndCreateCollectionTest, DeleteReplicaTest, DistribCursorPagingTest, MigrateRouteKeyTest, SSLMigrationTest, SimpleCollectionCreateDeleteTest, TestLeaderElectionZkExpiry, TestMiniSolrCloudCluster, TestReplicaProperties]
   [junit4] Completed [435/543] on J2 in 64.65s, 1 test, 1 failure <<< FAILURES!

[...truncated 363 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 1 failure, 31 ignored (18 assumptions) [seed: D08670B1FAA54301]

Total time: 204 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 999 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/999/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:58084/wb_dlc/o: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58084/wb_dlc/o: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=168934, name=coreZkRegister-6548-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=168934, name=coreZkRegister-6548-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([DC6E135E9ED086E8]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11261 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/init-core-data-001
   [junit4]   2> 3204461 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[DC6E135E9ED086E8]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /n/
   [junit4]   2> 3204465 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3204465 INFO  (Thread-159406) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3204465 INFO  (Thread-159406) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3204565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkTestServer start zk server on port:49762
   [junit4]   2> 3204565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204566 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204569 INFO  (zkCallback-2465-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11591a5e name:ZooKeeperConnection Watcher:127.0.0.1:49762 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3204573 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204574 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204575 INFO  (zkCallback-2466-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34d6f611 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204575 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204575 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204575 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3204577 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3204579 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3204580 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3204581 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3204582 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3204584 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3204584 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3204586 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3204587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3204588 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3204589 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3204590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3204590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3204592 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3204592 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3204594 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3204594 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3204596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3204596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3204598 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3204598 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3204600 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3204600 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3204602 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3204602 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3204915 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1
   [junit4]   2> 3204917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3204919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a2e0305{/n,null,AVAILABLE}
   [junit4]   2> 3204919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2c86b84{HTTP/1.1}{127.0.0.1:33937}
   [junit4]   2> 3204919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server Started @3207923ms
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores, hostContext=/n, hostPort=33937, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/tempDir-001/control/data}
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/'
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3204935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204937 INFO  (zkCallback-2467-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53e19508 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3204939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/solr.xml
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer New CoreContainer 608573074
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/]
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/lib
   [junit4]   2> 3204952 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/lib).
   [junit4]   2> 3204960 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3204961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3204961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3204961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49762/solr
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204964 INFO  (zkCallback-2469-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b2187f8 name:ZooKeeperConnection Watcher:127.0.0.1:49762 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204964 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204964 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204965 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204966 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6003761c name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204966 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3204969 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3204972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3204974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3204976 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3204977 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3204979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3204980 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3204981 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3204982 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33937_n
   [junit4]   2> 3204983 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33937_n
   [junit4]   2> 3204984 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3204985 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3204986 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3204987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94764414492934148-127.0.0.1:33937_n-n_0000000000
   [junit4]   2> 3204987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33937_n
   [junit4]   2> 3204987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3204988 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer Overseer (id=94764414492934148-127.0.0.1:33937_n-n_0000000000) starting
   [junit4]   2> 3204990 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3204996 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3204997 INFO  (OverseerCollectionConfigSetProcessor-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3204997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3204997 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3205012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3205012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3205013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores
   [junit4]   2> 3205014 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/, coreNodeName=, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1, name=collection1, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, loadOnStartup=true, schema=schema.xml, shard=, transient=false}
   [junit4]   2> 3205014 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/
   [junit4]   2> 3205014 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3205015 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3205015 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3205015 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3205015 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3205017 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"} current state version: 0
   [junit4]   2> 3205017 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"}
   [junit4]   2> 3205018 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3205018 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3205018 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3205019 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206018 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3206018 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/'
   [junit4]   2> 3206020 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3206020 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3206039 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3206048 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3206062 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3206128 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3206129 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3206137 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3206398 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3206406 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3206408 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3206417 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3206420 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3206423 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3206424 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3206424 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3206424 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3206425 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3206425 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3206426 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3206426 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3206426 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31bfe7
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3206427 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3206428 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/index
   [junit4]   2> 3206428 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 3206428 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4e6280cd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24f63aef),segFN=segments_1,generation=1}
   [junit4]   2> 3206429 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3206429 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/
   [junit4]   2> 3206429 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 3206432 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3206432 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3206434 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3206434 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3206434 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3206437 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206438 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206440 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206441 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206444 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/system,/update,/admin/segments,/admin/threads,/config,/admin/properties,/update/csv,/update/json/docs,standard,/admin/luke,/admin/plugins,/replication,/update/json,/admin/ping,/admin/mbeans,/admin/logging,/get,/schema,/admin/file
   [junit4]   2> 3206446 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3206446 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3206446 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3206447 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3206447 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=12, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4e6280cd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24f63aef),segFN=segments_1,generation=1}
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2a3f771[collection1] main
   [junit4]   2> 3206449 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3206449 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3206449 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3206451 INFO  (searcherExecutor-6500-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2a3f771[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3206452 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3206452 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3206452 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516230633968369664
   [junit4]   2> 3206454 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516230633968369664
   [junit4]   2> 3206454 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3206455 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3206455 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33937/n collection:control_collection shard:shard1
   [junit4]   2> 3206456 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3206459 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94764414492934148-core_node1-n_0000000000
   [junit4]   2> 3206460 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3206461 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33937/n/collection1/
   [junit4]   2> 3206462 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33937/n/collection1/ has no replicas
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33937/n/collection1/ shard1
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3206465 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3206468 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3206517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206519 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@570fd5c3 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206520 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3206522 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3206522 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3206522 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206523 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206524 INFO  (zkCallback-2473-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b2b1701 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206524 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206524 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206526 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 3206526 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3206527 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206530 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33937/n/collection1/ and leader is http://127.0.0.1:33937/n/collection1/
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33937/n
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3206569 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3206570 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"} current state version: 2
   [junit4]   2> 3206571 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"}
   [junit4]   2> 3206572 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3206676 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206676 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206852 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores/collection1
   [junit4]   2> 3206853 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001
   [junit4]   2> 3206854 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37a6cfcd{/n,null,AVAILABLE}
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3bce107a{HTTP/1.1}{127.0.0.1:49376}
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server Started @3209860ms
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/n, hostPort=49376, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores}
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/'
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3206873 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206874 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206875 INFO  (zkCallback-2474-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@382dca3d name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206875 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206875 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206876 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3206876 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/solr.xml
   [junit4]   2> 3206891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores
   [junit4]   2> 3206892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer New CoreContainer 794072415
   [junit4]   2> 3206892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/]
   [junit4]   2> 3206892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/lib
   [junit4]   2> 3206892 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/lib).
   [junit4]   2> 3206900 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49762/solr
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3206903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206904 INFO  (zkCallback-2476-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f0f36d5 name:ZooKeeperConnection Watcher:127.0.0.1:49762 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206905 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206905 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206907 INFO  (zkCallback-2477-thread-1-processing-n:127.0.0.1:49376_n) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a8baeb8 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206907 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3207920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49376_n
   [junit4]   2> 3207920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49376_n
   [junit4]   2> 3207922 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3207922 INFO  (zkCallback-2477-thread-1-processing-n:127.0.0.1:49376_n) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3207922 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3207924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3207924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94764414492934153-127.0.0.1:49376_n-n_0000000001
   [junit4]   2> 3207925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94764414492934148-127.0.0.1:33937_n-n_0000000000 to know if I could be the leader
   [junit4]   2> 3207937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3207937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3207938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores
   [junit4]   2> 3207939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, coreNodeName=, configSetProperties=configsetprops.json, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores/collection1, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores/collection1/, shard=, transient=false, config=solrconfig.xml, collection=collection1, loadOnStartup=true}
   [junit4]   2> 3207939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/c

[...truncated too long message...]

67/wb_dlc/o]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC6E135E9ED086E8]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53022/wb_dlc/o, http://127.0.0.1:57344/wb_dlc/o, http://127.0.0.1:43240/wb_dlc/o, http://127.0.0.1:58084/wb_dlc/o, http://127.0.0.1:33567/wb_dlc/o]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57344/wb_dlc/o
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 11 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9598, name=SyncThread:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.SecurityException: java.lang.System#exit(11) calls are not allowed because they terminate the test runner's JVM.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC6E135E9ED086E8]:0)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:86)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:53)
   [junit4]    > 	at java.security.AccessController.doPrivileged(Native Method)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager.checkExit(TestSecurityManager.java:53)
   [junit4]    > 	at java.lang.Runtime.exit(Runtime.java:107)
   [junit4]    > 	at java.lang.System.exit(System.java:962)
   [junit4]    > 	at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:185)
   [junit4]   2> 6611272 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[DC6E135E9ED086E8]-worker) [n:127.0.0.1:33567_wb_dlc%2Fo c:awholynewstresscollection_collection0_1 s:shard7 r:core_node18 x:awholynewstresscollection_collection0_1_shard7_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DC6E135E9ED086E8-001
   [junit4]   2> Oct 27, 2015 8:53:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {other_tl1=BlockTreeOrds(blocksize=128), multiDefault=PostingsFormat(name=Direct), range_facet_i_dv=PostingsFormat(name=Direct), range_facet_l_dv=Lucene50(blocksize=128), a_t=PostingsFormat(name=Direct), _version_=BlockTreeOrds(blocksize=128), text=PostingsFormat(name=LuceneVarGapDocFreqInterval), intDefault=BlockTreeOrds(blocksize=128), a_i1=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=Direct), id=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=Direct)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Lucene54), range_facet_l_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene54)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=th, timezone=Turkey
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=168141152,total=493355008
   [junit4]   2> NOTE: All tests run in this JVM: [RollingRestartTest, TestSolr4Spatial2, SpatialFilterTest, TestUpdate, AddBlockUpdateTest, TestConfigOverlay, AsyncCallRequestStatusResponseTest, TestHighlightDedupGrouping, DistributedFacetPivotLongTailTest, TestManagedSchemaFieldTypeResource, PKIAuthenticationIntegrationTest, TestConfigSetProperties, TestRuleBasedAuthorizationPlugin, TestFastOutputStream, MinimalSchemaTest, ChaosMonkeySafeLeaderTest, NotRequiredUniqueKeyTest, TestDistributedGrouping, SegmentsInfoRequestHandlerTest, UniqFieldsUpdateProcessorFactoryTest, CollectionStateFormat2Test, BJQParserTest, BigEndianAscendingWordSerializerTest, SaslZkACLProviderTest, TestReloadDeadlock, TestBlobHandler, SpellPossibilityIteratorTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [480/543] on J1 in 5182.97s, 1 test, 1 error <<< FAILURES!

[...truncated 198 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151027_190323_791.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18207.hprof ...
   [junit4] Heap dump file created [675873746 bytes in 6.622 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 11 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2146 tests, 2 errors, 31 ignored (18 assumptions) [seed: DC6E135E9ED086E8]

Total time: 260 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 998 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/998/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15048, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15048, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: collection already exists: awholynewstresscollection_collection3_4
	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=27952, name=coreZkRegister-2805-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=27952, name=coreZkRegister-2805-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10459 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/init-core-data-001
   [junit4]   2> 1042169 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[463320E19318F710]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1042175 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1042177 INFO  (Thread-22791) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1042177 INFO  (Thread-22791) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1042277 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkTestServer start zk server on port:60764
   [junit4]   2> 1042277 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1042278 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1042280 INFO  (zkCallback-1481-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a775add name:ZooKeeperConnection Watcher:127.0.0.1:60764 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1042280 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1042280 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1042280 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1042291 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1042291 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1042294 INFO  (zkCallback-1482-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f6dd026 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1042294 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1042294 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1042294 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1042297 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1042298 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1042302 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1042310 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1042310 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1042315 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1042316 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1042320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1042320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1042332 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1042332 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1042339 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1042339 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1042343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1042343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1042349 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1042350 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1042353 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1042353 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1042360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1042360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1042364 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1042364 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1042368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1042368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1042910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1
   [junit4]   2> 1042915 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1042927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4aeb148e{/,null,AVAILABLE}
   [junit4]   2> 1042927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.ServerConnector Started ServerConnector@f929d0f{HTTP/1.1}{127.0.0.1:36796}
   [junit4]   2> 1042927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server Started @1045824ms
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/tempDir-001/control/data, hostContext=/, hostPort=36796}
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/'
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1042952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1042957 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1042959 INFO  (zkCallback-1483-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@239fc541 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1042959 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1042960 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1042963 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1042963 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/solr.xml
   [junit4]   2> 1042994 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores
   [junit4]   2> 1042995 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer New CoreContainer 642035676
   [junit4]   2> 1042995 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/]
   [junit4]   2> 1042995 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/lib
   [junit4]   2> 1042995 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/lib).
   [junit4]   2> 1043017 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1043020 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1043021 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1043021 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1043023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60764/solr
   [junit4]   2> 1043023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1043023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1043024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1043027 INFO  (zkCallback-1485-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d0f2475 name:ZooKeeperConnection Watcher:127.0.0.1:60764 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1043030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1043031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1043035 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1043038 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74af7f8c name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1043038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1043039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1043043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1043057 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1043059 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1043063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1043066 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1043069 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1043070 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1043075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1043077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36796_
   [junit4]   2> 1043078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36796_
   [junit4]   2> 1043080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1043081 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1043083 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1043084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94758470776324100-127.0.0.1:36796_-n_0000000000
   [junit4]   2> 1043084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36796_
   [junit4]   2> 1043084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1043087 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer Overseer (id=94758470776324100-127.0.0.1:36796_-n_0000000000) starting
   [junit4]   2> 1043090 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1043105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1043105 INFO  (OverseerCollectionConfigSetProcessor-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1043105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1043108 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1043135 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1043135 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1043138 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores
   [junit4]   2> 1043139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, configSetProperties=configsetprops.json, shard=, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/, loadOnStartup=true, collection=control_collection, dataDir=data/, name=collection1, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1, schema=schema.xml}
   [junit4]   2> 1043139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/
   [junit4]   2> 1043139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1043144 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1043144 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1043146 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1043146 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1043151 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1043152 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1043152 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1043152 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1043153 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1043156 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1044153 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1044153 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1044153 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1044154 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/'
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1044193 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1044202 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1044258 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1044295 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1044298 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1044308 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1045882 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1045895 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1045901 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1045919 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1045922 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1045926 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1045927 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1045927 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1045928 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1045928 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1045929 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1045931 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ee6be14
   [junit4]   2> 1045932 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data
   [junit4]   2> 1045932 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1045932 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/index
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=9, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.28515625, noCFSRatio=0.42100909561013533]
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@74d03d00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@135506ea),segFN=segments_1,generation=1}
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1045945 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1045945 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1045945 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1045961 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1045963 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1045964 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/
   [junit4]   2> 1045965 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1045967 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1045989 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1046002 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/admin/luke,/admin/logging,/admin/plugins,/admin/properties,/config,/update/json/docs,/admin/system,/replication,/update/csv,/admin/threads,/update/json,standard,/admin/file,/admin/segments,/admin/mbeans,/get,/update,/admin/ping
   [junit4]   2> 1046017 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1046018 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1046018 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1046019 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1046019 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@74d03d00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@135506ea),segFN=segments_1,generation=1}
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@544c7732[collection1] main
   [junit4]   2> 1046021 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1046021 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1046021 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1046022 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1046022 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1046022 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1046023 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1046024 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1046024 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1046025 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1046025 INFO  (searcherExecutor-2757-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@544c7732[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1046028 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1046028 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1046028 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1046029 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516135536459251712
   [junit4]   2> 1046031 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516135536459251712
   [junit4]   2> 1046031 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1046033 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1046033 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:36796 collection:control_collection shard:shard1
   [junit4]   2> 1046034 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1046039 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94758470776324100-core_node1-n_0000000000
   [junit4]   2> 1046039 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36796/collection1/
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1046042 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36796/collection1/ has no replicas
   [junit4]   2> 1046042 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36796/collection1/ shard1
   [junit4]   2> 1046041 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1046045 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1046049 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046056 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1046064 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1046173 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:36796/collection1/ and leader is http://127.0.0.1:36796/collection1/
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36796
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1046220 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1046221 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1046233 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046235 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046241 INFO  (zkCallback-1488-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26670646 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046243 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046243 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046243 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1046247 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1046247 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1046247 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046281 INFO  (zkCallback-1489-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6add3c50 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046282 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046282 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046284 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1046284 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1046285 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046285 INFO  (zkCallback-1488-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046394 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1046789 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1
   [junit4]   2> 1046790 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001
   [junit4]   2> 1046802 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1046814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10e2510{/,null,AVAILABLE}
   [junit4]   2> 1046814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2e3a0cd4{HTTP/1.1}{127.0.0.1:41209}
   [junit4]   2> 1046814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server Started @1049711ms
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/tempDir-001/jetty1, hostPort=41209, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores}
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/'
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1046843 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046845 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046849 INFO  (zkCallback-1490-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78e433dc name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046853 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046860 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046874 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1046876 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/solr.xml
   [junit4]   2> 1046901 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores
   [junit4]   2> 1046903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer New CoreContainer 893896919
   [junit4]   2> 1046903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/]
   [junit4]   2> 1046903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/lib
   [junit4]   2> 1046903 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/lib).
   [junit4]   2> 1046915 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1046917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1046917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1046917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1046918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60764/solr
   [junit4]   2> 1046918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1046918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046921 INFO  (zkCallback-1492-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@da0cfce name:ZooKeeperConnection Watcher:127.0.0.1:60764 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046923 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046923 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046928 INFO  (zkCallback-1493-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d1d4b72 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046950 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1047963 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41209_
   [junit4]   2> 1047964 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41209_
   [junit4]   2> 1047965 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1047965 INFO  (zkCallback-1488-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1047966 INFO  (zkCallback-1493-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1047968 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1047968 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94758470776324105-127.0.0.1:41209_-n_0000000001
   [junit4]   2> 1047969 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94758470776324100-127.0.0.1:36796_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1048016 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1048016 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1048023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores
   [junit4]   2> 1048024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1/, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, schema=schema.xml, dataDir=data/, shard=, name=collection1, transient=false}
   [junit4]   2> 1048024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1/
   [junit4]   2> 1048024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1048027 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1048027 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1048031 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:41209_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41209",
   [junit4]   2>   "core":"collection1"} current state version: 4
   [junit4]   2> 1048032 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:41209_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41209",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1048032 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 1048032 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1048034 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1048034 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1048035 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 1 
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1048040 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1048040 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1048041 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1048041 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1048043 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1048043 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:

[...truncated too long message...]

.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15046, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: collection already exists: awholynewstresscollection_collection1_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15047, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: collection already exists: awholynewstresscollection_collection2_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15050, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: Could not find collection : awholynewstresscollection_collection5_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]   2> 1902553 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[463320E19318F710]-worker) [n:127.0.0.1:57793_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {intDefault=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=PostingsFormat(name=LuceneFixedGap), other_tl1=PostingsFormat(name=LuceneFixedGap), a_t=PostingsFormat(name=LuceneFixedGap), _version_=PostingsFormat(name=LuceneFixedGap), a_i1=PostingsFormat(name=LuceneFixedGap), range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=LuceneFixedGap), id=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneFixedGap)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=zh_TW, timezone=America/North_Dakota/Center
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=72746112,total=508559360
   [junit4]   2> NOTE: All tests run in this JVM: [AsyncCallRequestStatusResponseTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestRestoreCore, BasicFunctionalityTest, TestStandardQParsers, HdfsCollectionsAPIDistributedZkTest, SliceStateTest, TestDistributedGrouping, TestSuggestSpellingConverter, CustomCollectionTest, PreAnalyzedFieldTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestHdfsUpdateLog, TestManagedSchemaDynamicFieldResource, CircularListTest, DocumentBuilderTest, LoggingHandlerTest, TestSolr4Spatial2, TestImplicitCoreProperties, DistributedFacetPivotLongTailTest, DocValuesTest, SaslZkACLProviderTest, OverseerTaskQueueTest, SpellCheckComponentTest, CursorPagingTest, TestSortingResponseWriter, TestSolrCoreProperties, TestQueryTypes, HdfsNNFailoverTest, SimplePostToolTest, DOMUtilTest, HdfsSyncSliceTest, AnalyticsQueryTest, ConvertedLegacyTest, SolrPluginUtilsTest, TestNoOpRegenerator, MoreLikeThisHandlerTest, TestFastOutputStream, TestCrossCoreJoin, SpellCheckCollatorTest, OverseerStatusTest, BJQParserTest, TestFieldTypeResource, TermsComponentTest, TestCodecSupport, AtomicUpdatesTest, DistributedTermsComponentTest, TestStressVersions, SolrCmdDistributorTest, TestSolrDeletionPolicy2, TestUtils, TestJmxMonitoredMap, TestClusterStateMutator, RecoveryAfterSoftCommitTest, SuggesterFSTTest, TestConfigOverlay, SolrIndexSplitterTest, TestBulkSchemaConcurrent, TestDFRSimilarityFactory, TestTrieFacet, TestRandomRequestDistribution, SuggestComponentContextFilterQueryTest, URLClassifyProcessorTest, OpenCloseCoreStressTest, TestHighlightDedupGrouping, RequestLoggingTest, TestQuerySenderNoQuery, TestAnalyzeInfixSuggestions, TestLeaderInitiatedRecoveryThread, TestWordDelimiterFilterFactory, TestManagedSchemaFieldTypeResource, TestDefaultStatsCache, FastVectorHighlighterTest, TestRestManager, CollectionsAPIDistributedZkTest]
   [junit4] Completed [222/543] on J1 in 363.93s, 1 test, 1 error <<< FAILURES!

[...truncated 1059 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2144 tests, 2 errors, 31 ignored (18 assumptions) [seed: 463320E19318F710]

Total time: 227 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 997 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/997/

2 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}, {indexVersion=1445818330639,generation=3,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _5w.cfe, _5w.cfs, _5w.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}, {indexVersion=1445818330639,generation=3,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _5w.cfe, _5w.cfs, _5w.si, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([A6D967FEF6F05DCB:830E7CCE86B853C8]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=11480, name=coreZkRegister-3481-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11480, name=coreZkRegister-3481-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([A6D967FEF6F05DCB]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10787 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/init-core-data-001
   [junit4]   2> 1971994 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[A6D967FEF6F05DCB]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /j_/
   [junit4]   2> 1971998 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1971999 INFO  (Thread-4444) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1971999 INFO  (Thread-4444) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1972099 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkTestServer start zk server on port:55034
   [junit4]   2> 1972099 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972099 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972102 INFO  (zkCallback-1773-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@512a9ce7 name:ZooKeeperConnection Watcher:127.0.0.1:55034 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972102 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1972105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972106 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972107 INFO  (zkCallback-1774-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f80947e name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972107 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1972109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1972111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1972112 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1972114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1972114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1972117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1972117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1972119 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1972120 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1972122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1972122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1972124 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1972124 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1972126 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1972126 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1972129 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1972130 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1972131 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1972132 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1972139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1972139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1972141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1972141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1972142 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1972143 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1972431 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1
   [junit4]   2> 1972433 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7913eec{/j_,null,AVAILABLE}
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6244865a{HTTP/1.1}{127.0.0.1:47317}
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.Server Started @1975402ms
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/j_, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/tempDir-001/control/data, hostPort=47317, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores}
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/'
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1972453 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972453 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972455 INFO  (zkCallback-1775-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c83ceb name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972455 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972455 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972456 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1972456 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/solr.xml
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer New CoreContainer 1243423389
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/]
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/lib
   [junit4]   2> 1972470 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/lib).
   [junit4]   2> 1972478 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1972479 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55034/solr
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972488 INFO  (zkCallback-1777-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f1b0752 name:ZooKeeperConnection Watcher:127.0.0.1:55034 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972491 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972492 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@543d4d5f name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972492 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1972495 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1972500 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1972502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1972504 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1972506 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1972508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1972509 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1972510 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1972511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47317_j_
   [junit4]   2> 1972511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47317_j_
   [junit4]   2> 1972513 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1972514 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1972515 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1972516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94753102892564484-127.0.0.1:47317_j_-n_0000000000
   [junit4]   2> 1972516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47317_j_
   [junit4]   2> 1972516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1972518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer Overseer (id=94753102892564484-127.0.0.1:47317_j_-n_0000000000) starting
   [junit4]   2> 1972519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1972526 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1972526 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1972526 INFO  (OverseerCollectionConfigSetProcessor-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1972526 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1972547 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1972547 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1972548 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores
   [junit4]   2> 1972549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1, transient=false, shard=, name=collection1, config=solrconfig.xml, loadOnStartup=true, collection=control_collection, dataDir=data/, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/, configSetProperties=configsetprops.json}
   [junit4]   2> 1972549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/
   [junit4]   2> 1972549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1972551 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1972552 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1972552 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1972552 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1972554 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47317/j_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:47317_j_",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1972554 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1972554 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47317/j_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:47317_j_",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1972555 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1972555 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1972558 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1973555 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1973555 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/'
   [junit4]   2> 1973558 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1973558 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1973595 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1973603 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1973617 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1973677 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1973678 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1973685 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1974253 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1974262 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1974267 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1974281 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1974284 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1974294 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1974295 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1974295 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1974295 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1974297 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1974298 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a147b53
   [junit4]   2> 1974300 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data
   [junit4]   2> 1974308 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1974308 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1974308 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/index
   [junit4]   2> 1974309 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.2626953125, noCFSRatio=1.0]
   [junit4]   2> 1974309 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e942479 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3fe6bca6),segFN=segments_1,generation=1}
   [junit4]   2> 1974309 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1974325 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1974325 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1974325 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1974327 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/
   [junit4]   2> 1974331 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1974332 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974334 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974335 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974336 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974359 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,/admin/threads,/replication,/config,/admin/mbeans,/admin/properties,/get,standard,/admin/logging,/admin/ping,/admin/file,/update/json/docs,/admin/segments,/schema,/admin/system,/update/csv,/update,/admin/plugins,/admin/luke
   [junit4]   2> 1974367 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1974388 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1974388 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1974389 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1974389 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1974389 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1184a6ad
   [junit4]   2> 1974399 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e942479 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3fe6bca6),segFN=segments_1,generation=1}
   [junit4]   2> 1974399 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1974399 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@29ed5726[collection1] main
   [junit4]   2> 1974400 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1974423 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1974423 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1974424 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1974424 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1974424 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1974426 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1974426 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1974426 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516049648824877056
   [junit4]   2> 1974432 INFO  (searcherExecutor-3444-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@29ed5726[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1974433 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 7.0ms to seed version buckets with highest version 1516049648824877056
   [junit4]   2> 1974433 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1974440 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1974440 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:47317/j_ collection:control_collection shard:shard1
   [junit4]   2> 1974442 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1974446 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94753102892564484-core_node1-n_0000000000
   [junit4]   2> 1974447 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47317/j_/collection1/
   [junit4]   2> 1974449 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:47317/j_/collection1/ has no replicas
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47317/j_/collection1/ shard1
   [junit4]   2> 1974450 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1974453 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1974456 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1974458 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47317/j_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1974561 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1974562 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1974566 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1974568 INFO  (zkCallback-1780-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77d5e3e1 name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1974568 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1974569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1974569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1974577 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1974612 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:47317/j_/collection1/ and leader is http://127.0.0.1:47317/j_/collection1/
   [junit4]   2> 1974612 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:4731

[...truncated too long message...]

ndex
   [junit4]   2> 2700555 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-027/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-027/collection1/data;done=false>>]
   [junit4]   2> 2700555 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-027/collection1/data
   [junit4]   2> 2700555 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@31b1f6e6{/solr,null,UNAVAILABLE}
   [junit4]   2> 2700562 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@f0c3f60{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1239272902
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@9e5cb00
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data;done=false>>]
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data
   [junit4]   2> 2700567 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data/index;done=false>>]
   [junit4]   2> 2700567 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data/index
   [junit4]   2> 2700567 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@927b9a2{/solr,null,UNAVAILABLE}
   [junit4]   2> 2700569 INFO  (SUITE-TestReplicationHandler-seed#[A6D967FEF6F05DCB]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=en, timezone=America/Rankin_Inlet
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=233224352,total=457703424
   [junit4]   2> NOTE: All tests run in this JVM: [ReplicationFactorTest, IndexSchemaTest, TestPhraseSuggestions, BigEndianAscendingWordDeserializerTest, TestValueSourceCache, CleanupOldIndexTest, HdfsDirectoryFactoryTest, OverriddenZkACLAndCredentialsProvidersTest, TestDistributedStatsComponentCardinality, SynonymTokenizerTest, CollectionReloadTest, TestSolrJ, TestBinaryField, TestFoldingMultitermQuery, EchoParamsTest, TestDistributedSearch, ZkStateReaderTest, TestConfigSetsAPIExclusivity, TestConfigReload, TestOmitPositions, LeaderInitiatedRecoveryOnCommitTest, QueryElevationComponentTest, TestHighFrequencyDictionaryFactory, SignatureUpdateProcessorFactoryTest, PathHierarchyTokenizerFactoryTest, TestRemoteStreaming, TestTolerantSearch, TestIBSimilarityFactory, TestCollectionAPI, JavabinLoaderTest, TestBadConfig, TestRandomFaceting, TestDocumentBuilder, TestSearcherReuse, SolrCoreTest, TestJsonFacets, DistributedFacetPivotSmallAdvancedTest, TestLMDirichletSimilarityFactory, OverseerCollectionConfigSetProcessorTest, TestDistributedMissingSort, ResponseHeaderTest, HdfsBasicDistributedZk2Test, TestExclusionRuleCollectionAccess, HdfsUnloadDistributedZkTest, BasicAuthIntegrationTest, ReturnFieldsTest, TestFieldSortValues, PingRequestHandlerTest, DataDrivenBlockJoinTest, TestCollapseQParserPlugin, CloudExitableDirectoryReaderTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestCollationFieldDocValues, HighlighterMaxOffsetTest, TestMiniSolrCloudClusterBase, TestRandomDVFaceting, MigrateRouteKeyTest, TestRawTransformer, CacheHeaderTest, DisMaxRequestHandlerTest, SSLMigrationTest, ResourceLoaderTest, PrimUtilsTest, TestDocBasedVersionConstraints, TestManagedSynonymFilterFactory, FileUtilsTest, SyncSliceTest, SpellingQueryConverterTest, TestJoin, FullHLLTest, TestCustomSort, TestInitParams, DistributedQueryElevationComponentTest, TestConfig, TestAtomicUpdateErrorCases, OutputWriterTest, JsonLoaderTest, LeaderFailoverAfterPartitionTest, BasicDistributedZkTest, TestCloudSchemaless, StandardRequestHandlerTest, BadCopyFieldTest, TestSchemaSimilarityResource, IndexSchemaRuntimeFieldTest, TestRangeQuery, SolrCloudExampleTest, SolrTestCaseJ4Test, TestReplicationHandler]
   [junit4] Completed [396/543] on J0 in 660.67s, 13 tests, 1 failure <<< FAILURES!

[...truncated 492 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2144 tests, 1 error, 1 failure, 31 ignored (18 assumptions) [seed: A6D967FEF6F05DCB]

Total time: 201 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 996 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/996/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:38621: Could not load collection from ZK:acollectionafterbaddelete

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38621: Could not load collection from ZK:acollectionafterbaddelete
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=20121, name=coreZkRegister-6242-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20121, name=coreZkRegister-6242-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([6D2393130133B000]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior

Error Message:
Illegal state, was: down expected:active clusterState:live nodes:[]collections:{c1=DocCollection(c1)={   "router":{"name":"implicit"},   "shards":{"shard1":{       "parent":null,       "state":"active",       "range":null,       "replicas":{"core_node1":{           "base_url":"http://127.0.0.1/solr",           "node_name":"node1",           "core":"core1",           "roles":"",           "state":"down"}}}}}, test=LazyCollectionRef(test)}

Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
  "router":{"name":"implicit"},
  "shards":{"shard1":{
      "parent":null,
      "state":"active",
      "range":null,
      "replicas":{"core_node1":{
          "base_url":"http://127.0.0.1/solr",
          "node_name":"node1",
          "core":"core1",
          "roles":"",
          "state":"down"}}}}}, test=LazyCollectionRef(test)}
	at __randomizedtesting.SeedInfo.seed([6D2393130133B000:53D90FFE3A3EA4E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.OverseerTest.verifyStatus(OverseerTest.java:601)
	at org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1261)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10678 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/init-core-data-001
   [junit4]   2> 1285696 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6D2393130133B000]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1285696 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6D2393130133B000]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1285701 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1285701 INFO  (Thread-4477) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1285701 INFO  (Thread-4477) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1285801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkTestServer start zk server on port:44976
   [junit4]   2> 1285801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285804 INFO  (zkCallback-1000-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8910ea2 name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1285807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285808 INFO  (zkCallback-1001-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c375294 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285808 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1285810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1285812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1285813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1285815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1285815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1285817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1285817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1285819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285821 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1285821 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1285823 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1285823 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1285825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1285825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1285827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1285827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1285829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1285829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1285831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1285833 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1285834 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1285834 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1285836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285837 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285838 INFO  (zkCallback-1002-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66746ee1 name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1285840 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285840 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285842 INFO  (zkCallback-1003-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e58604f name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285842 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285842 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1285843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1285845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1285845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1285847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1285847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1285849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1285851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1285853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1285853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1285855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1285855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1285857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1285857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1285859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1285861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1285863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1285863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1285865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1285865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1286156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1
   [junit4]   2> 1286157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1286159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@73b4d45a{/,null,AVAILABLE}
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.ServerConnector Started ServerConnector@17cc8ee0{HTTP/1.1}{127.0.0.1:56839}
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server Started @1289063ms
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores, hostPort=56839, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/tempDir-001/control/data}
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/'
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1286177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1286178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1286179 INFO  (zkCallback-1004-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c0fecc4 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1286179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1286180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1286181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1286181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/solr.xml
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer New CoreContainer 120356151
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/]
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/lib
   [junit4]   2> 1286210 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/lib).
   [junit4]   2> 1286217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44976/solr
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1286220 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1286221 INFO  (zkCallback-1006-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@c984492 name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1286222 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1286222 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1286223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1286224 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f990515 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1286224 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1286226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1286228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1286232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1286234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1286236 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1286238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1286240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1286241 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1286243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1286244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56839_
   [junit4]   2> 1286244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56839_
   [junit4]   2> 1286246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1286248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1286249 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1286250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94747307604443142-127.0.0.1:56839_-n_0000000000
   [junit4]   2> 1286250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56839_
   [junit4]   2> 1286250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1286251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer Overseer (id=94747307604443142-127.0.0.1:56839_-n_0000000000) starting
   [junit4]   2> 1286253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1286259 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1286260 INFO  (OverseerCollectionConfigSetProcessor-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1286260 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1286260 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1286281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1286281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1286282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores
   [junit4]   2> 1286283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, transient=false, loadOnStartup=true, config=solrconfig.xml, collection=control_collection, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1, schema=schema.xml, shard=, name=collection1, dataDir=data/}
   [junit4]   2> 1286283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/
   [junit4]   2> 1286283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1286284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1286284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1286284 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1286284 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1286286 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1286286 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1286287 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1286287 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1286288 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1286289 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/'
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1287302 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1287307 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1287320 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1287348 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1287349 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1287355 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1288354 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1288362 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1288364 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1288373 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1288376 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1288379 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1288380 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1288381 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@238e338
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data
   [junit4]   2> 1288383 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1288383 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1288383 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/index
   [junit4]   2> 1288383 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5178545787741137]
   [junit4]   2> 1288384 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6d7e0d74 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6ea572d4),segFN=segments_1,generation=1}
   [junit4]   2> 1288384 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1288384 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/
   [junit4]   2> 1288384 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1288390 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1288390 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1288392 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288394 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288395 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288396 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288399 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/update,/schema,/get,/update/json/docs,/admin/properties,/admin/logging,/admin/plugins,/admin/mbeans,/admin/ping,/replication,/config,/admin/system,/update/csv,/admin/luke,/update/json,/admin/threads,/admin/file,standard
   [junit4]   2> 1288401 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1288401 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1288401 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1288402 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1288402 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1288402 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=62.6435546875, floorSegmentMB=0.212890625, forceMergeDeletesPctAllowed=27.910711213771886, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6d7e0d74 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6ea572d4),segFN=segments_1,generation=1}
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@59f3a728[collection1] main
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1288406 INFO  (searcherExecutor-2769-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59f3a728[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515956924506439680
   [junit4]   2> 1288408 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515956924506439680
   [junit4]   2> 1288408 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1288409 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1288409 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:56839 collection:control_collection shard:shard1
   [junit4]   2> 1288410 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1288414 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94747307604443142-core_node1-n_0000000000
   [junit4]   2> 1288414 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56839/collection1/
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1288417 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56839/collection1/ has no replicas
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56839/collection1/ shard1
   [junit4]   2> 1288418 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1288418 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1288420 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1288423 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1288525 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:56839/collection1/ and leader is http://127.0.0.1:56839/collection1/
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56839
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1288576 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1288576 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1288679 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1288739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1288740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1288742 INFO  (zkCallback-1009-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5faaf408 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1288742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1288742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1288742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1288745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1289054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/cores/collection1
   [junit4]   2> 1289055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001
   [junit4]   2> 1289056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1289058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@66a67b1{/,null,AVAILABLE}
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.ServerConnector Started ServerConnector@723cd397{HTTP/1.1}{127.0.0.1:38621}
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server Started @1291961ms
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/tempDir-001/jetty1, hostContext=/, solrconfig=solrconfig.xml, hostPort=38621, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/cores}
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/'
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1289076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1289077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1289079 INFO  (zkCallback-1010-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@570481dc name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1289079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1289079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1289080 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1289080 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/solr.xml
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/cores
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer New CoreContainer 1051817571
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/]
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/lib
   [junit4]   2> 1289096 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/lib).
   [junit4]   2> 1289104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1289105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44976/solr
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1289108 INFO  (zkCallback-1012-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@708c513d name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1289108 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1289108 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1289110 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1289111 INFO  (zkCallback-1013-thread-1-processing-n:127.0.0.1:38621_) [n:127.0.0.1:38621_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22ff011f name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1289111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1289118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1290124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38621_
   [junit4]   2> 1290125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38621_
   [junit4]   2> 1290126 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1290126 INFO  (zkCallback-1013-thread-1-processing-n:127.0.0.1:38621_) [n:127.0.0.1:38621_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 

[...truncated too long message...]

ildren watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=6D2393130133B000 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no_NO_NY -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    367s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20121, name=coreZkRegister-6242-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6D2393130133B000]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3452655 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[6D2393130133B000]-worker) [n:127.0.0.1:50123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6D2393130133B000-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=no_NO_NY, timezone=America/Halifax
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=288424312,total=526909440
   [junit4]   2> NOTE: All tests run in this JVM: [BinaryUpdateRequestHandlerTest, DistributedFacetPivotLargeTest, CachingDirectoryFactoryTest, TestConfigSetsAPIZkFailure, TestCloudManagedSchemaConcurrent, TestJmxIntegration, TestOrdValues, TestDynamicFieldCollectionResource, TestReRankQParserPlugin, TestReloadAndDeleteDocs, SuggesterTest, TestHashQParserPlugin, QueryResultKeyTest, TestSimpleTrackingShardHandler, DocValuesMultiTest, TestScoreJoinQPScore, TestRequestStatusCollectionAPI, TestRebalanceLeaders, SolrCoreCheckLockOnStartupTest, TestDefaultSearchFieldResource, BasicDistributedZk2Test, FieldMutatingUpdateProcessorTest, TestReload, DateRangeFieldTest, TestReversedWildcardFilterFactory, TestNRTOpen, TestElisionMultitermQuery, QueryParsingTest, TestXIncludeConfig, CurrencyFieldXmlFileTest, TestLRUCache, CollectionsAPIAsyncDistributedZkTest, BlockDirectoryTest, TestWriterPerf, RulesTest, TestBulkSchemaAPI, StatsComponentTest, FullSolrCloudDistribCmdsTest, DocExpirationUpdateProcessorFactoryTest, TestFuzzyAnalyzedSuggestions, StatelessScriptUpdateProcessorFactoryTest, DistributedSuggestComponentTest, TestSystemIdResolver, InfoHandlerTest, TestSort, RAMDirectoryFactoryTest, TestExceedMaxTermLength, BadComponentTest, HighlighterConfigTest, LeaderElectionTest, TestRemoveLastDynamicCopyField, AliasIntegrationTest, MergeStrategyTest, TestConfigSetImmutable, SampleTest, UUIDFieldTest, TestSmileRequest, TriLevelCompositeIdRoutingTest, AssignTest, AutoCommitTest, SparseHLLTest, TestLeaderElectionZkExpiry, DebugComponentTest, ScriptEngineTest, SuggesterTSTTest, TestSolrConfigHandlerConcurrent, XmlUpdateRequestHandlerTest, RuleEngineTest, DistributedQueryComponentCustomSortTest, OverseerRolesTest, TestSolrQueryParser, LukeRequestHandlerTest, DistributedExpandComponentTest, TestIntervalFaceting, ConcurrentDeleteAndCreateCollectionTest, DistribDocExpirationUpdateProcessorTest, FieldAnalysisRequestHandlerTest, TestChildDocTransformer, DirectUpdateHandlerTest, TestFastWriter, ShowFileRequestHandlerTest, TestEmbeddedSolrServerConstructors, UpdateParamsTest, TestFieldTypeCollectionResource, TestSolrIndexConfig, SearchHandlerTest, TestReplicaProperties, SimpleFacetsTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, TestRecovery, DistributedSpellCheckComponentTest, TestStressReorder, TestStressVersions, TestCoreContainer, TestSolr4Spatial, SolrCmdDistributorTest, PeerSyncTest, BadIndexSchemaTest, TestFunctionQuery, TestIndexSearcher, HighlighterTest, DistributedQueryElevationComponentTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, DocValuesTest, TestTrie, SpatialFilterTest, NoCacheHeaderTest, QueryEqualityTest, SortByFunctionTest, DistanceFunctionTest, SolrInfoMBeanTest, DisMaxRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, TestArbitraryIndexDir, TestCollationField, ReturnFieldsTest, JSONWriterTest, MBeansHandlerTest, UniqFieldsUpdateProcessorFactoryTest, CSVRequestHandlerTest, TestLFUCache, PreAnalyzedFieldTest, PrimUtilsTest, DOMUtilTest, ClusterStateTest, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SystemInfoHandlerTest, FileUtilsTest, TestRTGBase, DistributedIntervalFacetingTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, DeleteLastCustomShardedReplicaTest, DeleteShardTest, DistribCursorPagingTest, LeaderFailoverAfterPartitionTest, LeaderInitiatedRecoveryOnCommitTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [433/543] on J2 in 366.80s, 1 test, 1 error <<< FAILURES!

[...truncated 355 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151024_190452_256.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid19857.hprof ...
   [junit4] Heap dump file created [675741330 bytes in 11.260 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 15 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2144 tests, 2 errors, 1 failure, 31 ignored (18 assumptions) [seed: 6D2393130133B000]

Total time: 191 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 995 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/995/

5 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=484, name=coreZkRegister-215-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=484, name=coreZkRegister-215-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
ERROR: SolrIndexSearcher opens=51 closes=50

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=51 closes=50
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:468)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
file handle leaks: [FileChannel(/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-009/write.lock)]

Stack Trace:
java.lang.RuntimeException: file handle leaks: [FileChannel(/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-009/write.lock)]
	at org.apache.lucene.mockfile.LeakFS.onClose(LeakFS.java:64)
	at org.apache.lucene.mockfile.FilterFileSystem.close(FilterFileSystem.java:78)
	at org.apache.lucene.mockfile.FilterFileSystem.close(FilterFileSystem.java:79)
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:230)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception
	at org.apache.lucene.mockfile.LeakFS.onOpen(LeakFS.java:47)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:198)
	at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:167)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:334)
	at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:113)
	at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
	at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
	at org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:94)
	at org.apache.lucene.store.MockDirectoryWrapper.obtainLock(MockDirectoryWrapper.java:985)
	at org.apache.lucene.store.SleepingLockWrapper.obtainLock(SleepingLockWrapper.java:84)
	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:775)
	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:78)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:231)
	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:105)
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1784)
	at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:927)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:804)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:672)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:729)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:449)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:440)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)




Build Log:
[...truncated 9891 lines...]
   [junit4] Suite: org.apache.solr.core.TestLazyCores
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/init-core-data-001
   [junit4]   2> 11770 INFO  (SUITE-TestLazyCores-seed#[604EA955F55DFDE5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 11790 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testRace
   [junit4]   2> 11848 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/'
   [junit4]   2> 11848 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11848 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 11875 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer New CoreContainer 982569151
   [junit4]   2> 11875 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/]
   [junit4]   2> 11875 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/lib
   [junit4]   2> 11875 WARN  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/lib).
   [junit4]   2> 11914 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 11928 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 11929 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Allocating transient cache for 4 transient cores
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 11933 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, schema=schema.xml, dataDir=data/, instanceDir=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/, transient=false, name=collection1, config=solrconfig.xml, loadOnStartup=true}
   [junit4]   2> 11933 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/, dataDir=data/, transient=true, config=solrconfig.xml, instanceDir=collection2, loadOnStartup=true, name=collectionLazy2, configSetProperties=configsetprops.json, schema=schema.xml}
   [junit4]   2> 11934 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/, configSetProperties=configsetprops.json, instanceDir=collection3, schema=schema.xml, loadOnStartup=false, transient=on, name=collectionLazy3}
   [junit4]   2> 11934 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=collection4, name=collectionLazy4, loadOnStartup=false, configSetProperties=configsetprops.json, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection4/, dataDir=data/, transient=false, config=solrconfig.xml}
   [junit4]   2> 11935 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=collection5, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/, config=solrconfig.xml, name=collectionLazy5, loadOnStartup=true, dataDir=data/}
   [junit4]   2> 11935 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy6, instanceDir=collection6, transient=true, loadOnStartup=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection6/, configSetProperties=configsetprops.json, schema=schema.xml, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 11936 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy7, schema=schema.xml, dataDir=data/, transient=true, configSetProperties=configsetprops.json, config=solrconfig.xml, loadOnStartup=false, instanceDir=collection7, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection7/}
   [junit4]   2> 11936 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy8, configSetProperties=configsetprops.json, transient=true, schema=schema.xml, instanceDir=collection8, loadOnStartup=false, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection8/, config=solrconfig.xml}
   [junit4]   2> 11937 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, transient=true, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection9/, dataDir=data/, configSetProperties=configsetprops.json, name=collectionLazy9, loadOnStartup=false, instanceDir=collection9}
   [junit4]   2> 11938 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/'
   [junit4]   2> 11942 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11942 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 11947 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/'
   [junit4]   2> 11949 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/'
   [junit4]   2> 11949 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11949 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 11954 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11954 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 12045 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12053 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12079 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12103 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12138 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12150 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12241 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12241 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf/schema.xml
   [junit4]   2> 12250 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12251 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf/schema.xml
   [junit4]   2> 12267 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.s.IndexSchema [collectionLazy2] Schema name=tiny
   [junit4]   2> 12299 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=tiny
   [junit4]   2> 12337 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12339 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12339 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf/schema.xml
   [junit4]   2> 12351 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12351 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.s.IndexSchema [collectionLazy5] Schema name=tiny
   [junit4]   2> 12353 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf'
   [junit4]   2> 12353 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/
   [junit4]   2> 12353 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12379 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/], dataDir=[null]
   [junit4]   2> 12379 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 12380 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data
   [junit4]   2> 12380 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index/
   [junit4]   2> 12380 WARN  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12388 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf'
   [junit4]   2> 12388 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy2' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/
   [junit4]   2> 12388 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12389 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore [[collectionLazy2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/], dataDir=[null]
   [junit4]   2> 12389 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy2
   [junit4]   2> 12400 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data
   [junit4]   2> 12401 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index/
   [junit4]   2> 12401 WARN  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12407 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index
   [junit4]   2> 12408 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12401 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index
   [junit4]   2> 12408 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12408 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12410 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf'
   [junit4]   2> 12410 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/
   [junit4]   2> 12410 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12411 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore [[collectionLazy5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/], dataDir=[null]
   [junit4]   2> 12411 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy5
   [junit4]   2> 12414 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data
   [junit4]   2> 12414 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index/
   [junit4]   2> 12414 WARN  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12420 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index
   [junit4]   2> 12420 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12423 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12424 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12424 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12424 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12426 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-003 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12426 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12436 INFO  (OldIndexDirectoryCleanupThreadForCore-collectionLazy2) [    x:collectionLazy2] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collectionLazy2 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/
   [junit4]   2> 12437 WARN  (OldIndexDirectoryCleanupThreadForCore-collectionLazy2) [    x:collectionLazy2] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12453 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12453 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/
   [junit4]   2> 12454 INFO  (OldIndexDirectoryCleanupThreadForCore-collectionLazy5) [    x:collectionLazy5] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collectionLazy5 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/
   [junit4]   2> 12457 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12458 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12460 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12460 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12454 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12454 WARN  (OldIndexDirectoryCleanupThreadForCore-collectionLazy5) [    x:collectionLazy5] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12463 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12463 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12465 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12465 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12467 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12468 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12476 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12482 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12490 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12492 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12503 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12505 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12518 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12518 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12520 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 12537 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12538 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12539 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12540 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12540 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12540 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 12545 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12547 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12547 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12547 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.s.SolrIndexSearcher Opening Searcher@64d2da85[collectionLazy2] main
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3872f378[collection1] main
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf}
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 12595 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CoreContainer Opening transient core collectionLazy2
   [junit4]   2> 12595 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CoreContainer registering core: collectionLazy2
   [junit4]   2> 12596 INFO  (searcherExecutor-29-thread-1-processing-x:collectionLazy2) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2] Registered new searcher Searcher@64d2da85[collectionLazy2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf}
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 12597 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 12600 INFO  (searcherExecutor-28-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3872f378[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 12602 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-003 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12602 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12602 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.s.SolrIndexSearcher Opening Searcher@4a7437e4[collectionLazy5] main
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf}
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 12604 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.CoreContainer registering core: collectionLazy5
   [junit4]   2> 12604 INFO  (searcherExecutor-30-thread-1-processing-x:collectionLazy5) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5] Registered new searcher Searcher@4a7437e4[collectionLazy5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 12634 INFO  (Thread-14) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/'
   [junit4]   2> 12634 INFO  (Thread-14) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 12634 INFO  (Thread-14) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 12729 INFO  (Thread-14) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12754 INFO  (Thread-14) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12885 INFO  (Thread-14) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12885 INFO  (Thread-14) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf/schema.xml
   [junit4]   2> 12894 INFO  (Thread-14) [    ] o.a.s.s.IndexSchema [collectionLazy3] Schema name=tiny
   [junit4]   2> 12912 INFO  (Thread-14) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12914 INFO  (Thread-14) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf'
   [junit4]   2> 12914 INFO  (Thread-14) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy3' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/
   [junit4]   2> 12914 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12914 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore [[collectionLazy3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/], dataDir=[null]
   [junit4]   2> 12914 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy3
   [junit4]   2> 12915 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data
   [junit4]   2> 12915 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index/
   [junit4]   2> 12915 WARN  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12916 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index
   [junit4]   2> 12917 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12920 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12920 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12937 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12940 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12942 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12944 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12945 INFO  (OldIndexDirectoryCleanupThreadForCore-collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collectionLazy3 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/
   [junit4]   2> 12946 WARN  (OldIndexDirectoryCleanupThreadForCore-collectionLazy3) [    x:collectionLazy3] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12948 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12958 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 13014 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 13014 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.s.SolrIndexSearcher Opening Searcher@1886ad24[collectionLazy3] main
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf}
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 13016 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 13016 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CoreContainer Opening transient core collectionLazy3
   [junit4]   2> 13016 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CoreContainer registering core: collectionLazy3
   [junit4]   2> 13017 INFO  (searcherExecutor-40-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3] Registered new searcher Searcher@1886ad24[collectionLazy3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 13019 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=982569151
   [junit4]   2> 13019 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1b906ce8
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index;done=false>>]
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data;done=false>>]
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5]  CLOSING SolrCore org.apache.solr.core.SolrCore@6cec911f
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13029 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5] Closing main searcher on request.
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data;done=false>>]
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index;done=false>>]
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2]  CLOSING SolrCore org.apache.solr.core.SolrCore@34105c79
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2] Closing main searcher on request.
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index;done=false>>]
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data;done=false>>]
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3]  CLOSING SolrCore org.apache.solr.core.SolrCore@4ffc341f
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13035 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3] Closing main searcher on request.
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data;done=false>>]
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index;done=false>>]
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.SolrTestCaseJ4 ###Ending testRace
   [junit4]   2> 13059 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.SolrTestCaseJ4 ###Starting testLazySearch
   [junit4]   2> 13128 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/'
   [junit4]   2> 13128 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13128 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13151 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer New CoreContainer 1680872867
   [junit4]   2> 13152 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/]
   [junit4]   2> 13152 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/lib
   [junit4]   2> 13152 WARN  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/lib).
   [junit4]   2> 13165 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 13167 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer Allocating transient cache for 4 transient cores
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 13170 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, config=solrconfig.xml, dataDir=data/, instanceDir=collection1, loadOnStartup=true, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection1/, transient=false, configSetProperties=configsetprops.json}
   [junit4]   2> 13170 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, name=collectionLazy2, schema=schema.xml, dataDir=data/, instanceDir=collection2, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection2/, transient=true}
   [junit4]   2> 13171 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, instanceDir=collection3, schema=schema.xml, loadOnStartup=false, configSetProperties=configsetprops.json, name=collectionLazy3, transient=on, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection3/}
   [junit4]   2> 13171 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=collection4, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection4/, dataDir=data/, loadOnStartup=false, schema=schema.xml, name=collectionLazy4, config=solrconfig.xml, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 13172 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy5, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, instanceDir=collection5, configSetProperties=configsetprops.json, dataDir=data/, transient=false}
   [junit4]   2> 13172 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=false, config=solrconfig.xml, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection6/, name=collectionLazy6, transient=true, schema=schema.xml, configSetProperties=configsetprops.json, instanceDir=collection6}
   [junit4]   2> 13172 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, name=collectionLazy7, transient=true, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection7/, config=solrconfig.xml, instanceDir=collection7, dataDir=data/, loadOnStartup=false}
   [junit4]   2> 13173 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection8/, config=solrconfig.xml, loadOnStartup=false, name=collectionLazy8, schema=schema.xml, dataDir=data/, transient=true, instanceDir=collection8}
   [junit4]   2> 13173 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy9, schema=schema.xml, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection9/, instanceDir=collection9, loadOnStartup=false, dataDir=data/, config=solrconfig.xml, transient=true}
   [junit4]   2> 13174 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection1/'
   [junit4]   2> 13178 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection2/'
   [junit4]   2> 13178 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13178 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13177 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/'
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13264 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 13271 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 13279 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 13314 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 13322 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 13348 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 13426 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 13426 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/conf/schema.xml
   [junit4]   2> 13449 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema [collectionLazy5] Schema name=tiny
   [junit4]   2> 13452 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 13452 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection2/conf/schema.xml
   [junit4]   2> 13471 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 13472 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection1/conf/schema.xml
   [junit4]   2> 13475 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema [collectionLazy2] Schema name=tiny
   [junit4]   2> 13478 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 13480 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/conf'
   [junit4]   2> 13480 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/
   [junit4]   2> 13480 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 13481 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c.SolrCore [[collectionLazy5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/], dataDir=[null]
   [junit4]   2> 13481 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy5
   [junit4]   2> 13482 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c

[...truncated too long message...]

onExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election/94741961935421458-core_node2-n_0000000003
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:166)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:268)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:450)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 434742 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[604EA955F55DFDE5]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@1d923336{HTTP/1.1}{127.0.0.1:37038}
   [junit4]   2> 434743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[604EA955F55DFDE5]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@48bc20ba{/mcfnu/hs,null,UNAVAILABLE}
   [junit4]   2> 434745 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[604EA955F55DFDE5]) [n:127.0.0.1:37038_mcfnu%2Fhs c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40546 40546
   [junit4]   2> 434907 INFO  (Thread-112) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40546 40546
   [junit4]   2> 434910 WARN  (Thread-112) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/aliases.json
   [junit4]   2> 	6	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	36	/solr/clusterstate.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94741961935421444-127.0.0.1:45521_mcfnu%2Fhs-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/live_nodes
   [junit4]   2> 	8	/solr/collections
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=604EA955F55DFDE5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=be -Dtests.timezone=America/Dawson_Creek -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    375s J1 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=484, name=coreZkRegister-215-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 434943 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[604EA955F55DFDE5]-worker) [n:127.0.0.1:37038_mcfnu%2Fhs c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_604EA955F55DFDE5-001
   [junit4]   2> Oct 23, 2015 8:53:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=be, timezone=America/Dawson_Creek
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=103223288,total=343932928
   [junit4]   2> NOTE: All tests run in this JVM: [DistanceFunctionTest, DocValuesTest, TestReplicaProperties, TestNRTOpen, TestUpdate, TestBM25SimilarityFactory, PreAnalyzedFieldTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [406/542] on J1 in 375.63s, 1 test, 1 error <<< FAILURES!

[...truncated 430 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 4 suite-level errors, 1 error, 1449 ignored (7 assumptions) [seed: 604EA955F55DFDE5]

Total time: 165 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 994 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/994/

2 tests failed.
FAILED:  org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat.testRamBytesUsed

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)




Build Log:
[...truncated 5140 lines...]
   [junit4] Suite: org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat
   [junit4]   2> X 23, 2015 10:37:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat
   [junit4]   2>    1) Thread[id=144, name=SUITE-TestLucene42TermVectorsFormat-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:676)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:587)
   [junit4]   2>    2) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:49)
   [junit4]   2>    3) Thread[id=145, name=TEST-TestLucene42TermVectorsFormat.testRamBytesUsed-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2>         at org.apache.lucene.util.packed.BlockPackedReaderIterator.skip(BlockPackedReaderIterator.java:145)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.readPositions(Lucene42TermVectorsReader.java:584)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.get(Lucene42TermVectorsReader.java:398)
   [junit4]   2>         at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:194)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:187)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:127)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]   2>         at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:262)
   [junit4]   2>         at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:74)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    4) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:597)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:446)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene42TermVectorsFormat -Dtests.method=testRamBytesUsed -Dtests.seed=57A52995A507B6FB -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=bg_BG -Dtests.timezone=ACT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   7200s J0 | TestLucene42TermVectorsFormat.testRamBytesUsed <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)
   [junit4]   2> X 23, 2015 10:37:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> X 23, 2015 10:38:17 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat: 
   [junit4]   2>    1) Thread[id=145, name=TEST-TestLucene42TermVectorsFormat.testRamBytesUsed-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2>         at org.apache.lucene.store.MockIndexInputWrapper.readByte(MockIndexInputWrapper.java:132)
   [junit4]   2>         at org.apache.lucene.util.packed.BlockPackedReaderIterator.skip(BlockPackedReaderIterator.java:126)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.readPositions(Lucene42TermVectorsReader.java:584)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.get(Lucene42TermVectorsReader.java:398)
   [junit4]   2>         at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:194)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:187)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:127)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]   2>         at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:262)
   [junit4]   2>         at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:74)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> X 23, 2015 10:38:17 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=145, name=TEST-TestLucene42TermVectorsFormat.testRamBytesUsed-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene42.TestLucene42TermVectorsFormat_57A52995A507B6FB-001
   [junit4]   2> Oct 22, 2015 9:08:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Lucene53, sim=DefaultSimilarity, locale=bg_BG, timezone=ACT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=210661928,total=511180800
   [junit4]   2> NOTE: All tests run in this JVM: [TestLucene40TermVectorsFormat, TestLucene46UndeadNorms, TestLucene41PostingsFormat2, TestLucene46FieldInfoFormat, TestReuseDocsEnum, TestLucene42TermVectorsFormat]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene42TermVectorsFormat -Dtests.seed=57A52995A507B6FB -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=bg_BG -Dtests.timezone=ACT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | TestLucene42TermVectorsFormat (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)
   [junit4] Completed [30/30] on J0 in 7221.92s, 6 tests, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 30 suites, 789 tests, 1 suite-level error, 1 error, 81 ignored (81 assumptions) [seed: 57A52995A507B6FB]

Total time: 228 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 993 - Still Failing

Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
This is a valid bug caused by assertions added in SOLR-8069. See
https://issues.apache.org/jira/browse/SOLR-8069?focusedCommentId=14968664&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14968664

On Thu, Oct 22, 2015 at 12:17 PM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/993/
>
> 1 tests failed.
> FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR
>
> Error Message:
> Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
> Caused by: java.lang.AssertionError
>         at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
>         at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
>         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
>         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
>         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
>         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
>         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
>         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
>         at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
>         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
>         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 11227 lines...]
>    [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
>    [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/init-core-data-001
>    [junit4]   2> 2804605 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
>    [junit4]   2> 2804608 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
>    [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
>    [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer Starting server
>    [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer start zk server on port:52462
>    [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2804712 INFO  (zkCallback-2280-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a70d2b6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
>    [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2804716 INFO  (zkCallback-2281-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5128e657 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
>    [junit4]   2> 2804718 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
>    [junit4]   2> 2804719 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
>    [junit4]   2> 2804720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
>    [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
>    [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
>    [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
>    [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
>    [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
>    [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
>    [junit4]   2> 2804729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
>    [junit4]   2> 2804730 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
>    [junit4]   2> 2804731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
>    [junit4]   2> 2804732 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
>    [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
>    [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
>    [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 2804739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 2804740 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
>    [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
>    [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
>    [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
>    [junit4]   2> 2805008 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1
>    [junit4]   2> 2805010 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
>    [junit4]   2> 2805012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@665b505c{/,null,AVAILABLE}
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68a3874{HTTP/1.1}{127.0.0.1:56746}
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2807895ms
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=56746, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/control/data, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores}
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/'
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2805027 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2805028 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2805029 INFO  (zkCallback-2282-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39baa32 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
>    [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/solr.xml
>    [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
>    [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1984045032
>    [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/]
>    [junit4]   2> 2805044 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib
>    [junit4]   2> 2805044 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib).
>    [junit4]   2> 2805051 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2805054 INFO  (zkCallback-2284-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2e9d74e9 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2805055 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2805056 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2805057 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51ed1947 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
>    [junit4]   2> 2805061 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
>    [junit4]   2> 2805063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
>    [junit4]   2> 2805065 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
>    [junit4]   2> 2805066 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
>    [junit4]   2> 2805068 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
>    [junit4]   2> 2805069 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
>    [junit4]   2> 2805070 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
>    [junit4]   2> 2805071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
>    [junit4]   2> 2805072 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56746_
>    [junit4]   2> 2805073 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56746_
>    [junit4]   2> 2805074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
>    [junit4]   2> 2805075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
>    [junit4]   2> 2805076 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000
>    [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56746_
>    [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
>    [junit4]   2> 2805078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=94731933327425540-127.0.0.1:56746_-n_0000000000) starting
>    [junit4]   2> 2805080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
>    [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
>    [junit4]   2> 2805085 INFO  (OverseerCollectionConfigSetProcessor-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
>    [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 2805085 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Starting to work on the main queue
>    [junit4]   2> 2805108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
>    [junit4]   2> 2805109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer No authentication plugin used.
>    [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
>    [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, coreNodeName=, dataDir=data/, name=collection1, config=solrconfig.xml, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1, shard=, transient=false, schema=schema.xml}
>    [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/
>    [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found 1 core definitions
>    [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
>    [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
>    [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
>    [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
>    [junit4]   2> 2805113 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 0
>    [junit4]   2> 2805113 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
>    [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"}
>    [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
>    [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
>    [junit4]   2> 2805116 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
>    [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
>    [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
>    [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
>    [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/'
>    [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2806129 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0
>    [junit4]   2> 2806133 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
>    [junit4]   2> 2806145 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
>    [junit4]   2> 2806170 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
>    [junit4]   2> 2806171 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
>    [junit4]   2> 2806177 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
>    [junit4]   2> 2806414 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2806439 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
>    [junit4]   2> 2806440 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
>    [junit4]   2> 2806448 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
>    [junit4]   2> 2806451 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
>    [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
>    [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
>    [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
>    [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/], dataDir=[null]
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@379f19b4
>    [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data
>    [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index/
>    [junit4]   2> 2806457 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
>    [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index
>    [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7cd8dcec
>    [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
>    [junit4]   2>        commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
>    [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
>    [junit4]   2> 2806458 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/
>    [junit4]   2> 2806459 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2806463 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
>    [junit4]   2> 2806465 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806466 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806468 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806469 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806472 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/update/json,/admin/segments,/admin/mbeans,/admin/ping,/update/csv,/admin/plugins,/update,/admin/file,/replication,standard,/update/json/docs,/admin/threads,/admin/properties,/admin/luke,/admin/system,/get,/schema,/config
>    [junit4]   2> 2806473 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
>    [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
>    [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
>    [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=82.4736328125, floorSegmentMB=1.5283203125, forceMergeDeletesPctAllowed=4.112648727330555, segmentsPerTier=14.0, maxCFSSegmentSizeMB=1.6865234375, noCFSRatio=0.0
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
>    [junit4]   2>        commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@772028e6[collection1] main
>    [junit4]   2> 2806477 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
>    [junit4]   2> 2806479 INFO  (searcherExecutor-5949-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@772028e6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
>    [junit4]   2> 2806480 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515710935199645696
>    [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515710935199645696
>    [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
>    [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
>    [junit4]   2> 2806482 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:56746 collection:control_collection shard:shard1
>    [junit4]   2> 2806483 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
>    [junit4]   2> 2806486 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94731933327425540-core_node1-n_0000000000
>    [junit4]   2> 2806487 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56746/collection1/
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
>    [junit4]   2> 2806488 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection"} current state version: 1
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56746/collection1/ has no replicas
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56746/collection1/ shard1
>    [junit4]   2> 2806489 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
>    [junit4]   2> 2806491 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
>    [junit4]   2> 2806493 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "state":"active"} current state version: 1
>    [junit4]   2> 2806595 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806614 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806615 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@794415fc name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
>    [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
>    [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806618 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806619 INFO  (zkCallback-2288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77c64dbb name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"create",
>    [junit4]   2>   "name":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "stateFormat":"2"} current state version: 2
>    [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:56746/collection1/ and leader is http://127.0.0.1:56746/collection1/
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56746
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
>    [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "core_node_name":"core_node1",
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"active",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 2
>    [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "core_node_name":"core_node1",
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"active",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"}
>    [junit4]   2> 2806647 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
>    [junit4]   2> 2806750 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806750 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1
>    [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001
>    [junit4]   2> 2806892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@719f3138{/,null,AVAILABLE}
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3bc61657{HTTP/1.1}{127.0.0.1:50535}
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2809776ms
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/jetty1, hostContext=/, solrconfig=solrconfig.xml, hostPort=50535}
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/'
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2806909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806911 INFO  (zkCallback-2289-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@306792d6 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
>    [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/solr.xml
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1559122149
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/]
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib
>    [junit4]   2> 2806926 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib).
>    [junit4]   2> 2806934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806937 INFO  (zkCallback-2291-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c57b1e6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806940 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806941 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@124679be name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806941 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806945 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50535_
>    [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50535_
>    [junit4]   2> 2807952 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2807952 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2807952 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2807954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425545-127.0.0.1:50535_-n_0000000001
>    [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000 to know if I could be the leader
>    [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
>    [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer No authentication plugin used.
>    [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
>    [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1, name=collection1, configSetProperties=configsetprops.json, shard=, coreNodeName=, loadOnStartup=true, dataDir=data/, schema=schema.xml, transient=false, config=solrconfig.xml}
>    [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/
>    [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found 1 core definitions
>    [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
>    [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
>    [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
>    [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
>    [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
>    [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:50535",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"collection1",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:50535_"} current state version: 3
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:50535",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"collection1",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:50535_"}
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
>    [junit4]   2> 2807976 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0
>    [junit4]   2> 2807977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
>    [junit4]   2> 2808078 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
>    [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
>    [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1
>    [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
>    [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/'
>    [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2808992 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0
>    [junit4]   2> 2808997 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
>    [junit4]   2> 2809010 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
>    [junit4]   2> 2809039 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
>    [junit4]   2> 2809040 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
>    [junit4]   2> 2809046 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
>    [junit4]   2> 2809381 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2809388 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
>    [junit4]   2> 2809390 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
>    [junit4]   2> 2809400 INFO  (coreLoadExecutor-5959-thread-
>
> [...truncated too long message...]
>
> ill nodes recoverying - waited for 330 seconds
>    [junit4]    >        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
>    [junit4]    >        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
>    [junit4]    >        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
>    [junit4]    >        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
>    [junit4]    >        at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
>    [junit4]    >        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
>    [junit4]    >        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
>    [junit4]    > Caused by: java.lang.AssertionError
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
>    [junit4]    >        at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
>    [junit4]    >        at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
>    [junit4]    >        at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
>    [junit4]    >        at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
>    [junit4]    >        at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
>    [junit4]    >        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>    [junit4]    >        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>    [junit4]    >        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> 3169248 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [n:127.0.0.1:33453_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
>    [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001
>    [junit4]   2> Oct 22, 2015 2:22:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
>    [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=be_BY, timezone=Pacific/Port_Moresby
>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=204385304,total=520093696
>    [junit4]   2> NOTE: All tests run in this JVM: [TestExceedMaxTermLength, BinaryUpdateRequestHandlerTest, TestAuthorizationFramework, TestNamedUpdateProcessors, CoreAdminCreateDiscoverTest, TestCloudManagedSchema, TestRemoveLastDynamicCopyField, TestRawResponseWriter, TestCloudSchemaless, ZkStateReaderTest, TestZkChroot, TestSolrQueryParser, TestReload, TestReversedWildcardFilterFactory, HttpPartitionTest, TestSchemaManager, TestPostingsSolrHighlighter, HdfsLockFactoryTest, QueryParsingTest, TestIndexingPerformance, TestCloudManagedSchemaConcurrent, RAMDirectoryFactoryTest, CloudExitableDirectoryReaderTest, CoreAdminHandlerTest, VMParamsZkACLAndCredentialsProvidersTest, SampleTest, AutoCommitTest, TestSolrDeletionPolicy1, StatsComponentTest, SolrCoreCheckLockOnStartupTest, AsyncMigrateRouteKeyTest, TestSimpleQParserPlugin, SimpleCollectionCreateDeleteTest, SuggesterWFSTTest, VersionInfoTest, ChangedSchemaMergeTest, TestPKIAuthenticationPlugin, TestIndexSearcher, DistributedQueryComponentCustomSortTest, TestMacros, PrimitiveFieldTypeTest, DeleteShardTest, PluginInfoTest, HdfsRecoveryZkTest, SSLMigrationTest, DistributedFacetPivotSmallAdvancedTest, RequestHandlersTest, BadComponentTest, TestSearcherReuse, DocValuesMultiTest, ConnectionManagerTest, FullSolrCloudDistribCmdsTest, AnalyticsMergeStrategyTest, DeleteLastCustomShardedReplicaTest, TestConfigSets, TestCustomSort, TestMaxScoreQueryParser, CollectionReloadTest, TestInitParams, ExitableDirectoryReaderTest, DistribCursorPagingTest, DocValuesMissingTest, DebugComponentTest, TestSystemIdResolver, AnalysisErrorHandlingTest, TestSortByMinMaxFunction, DirectUpdateHandlerOptimizeTest, TestFieldTypeCollectionResource, SearchHandlerTest, TestElisionMultitermQuery, TestManagedStopFilterFactory, TestJmxIntegration, RulesTest, HighlighterMaxOffsetTest, TestCollationField, LukeRequestHandlerTest, TestDocBasedVersionConstraints, TestDefaultSearchFieldResource, FieldAnalysisRequestHandlerTest, FieldMutatingUpdateProcessorTest, TestPivotHelperCode, DocExpirationUpdateProcessorFactoryTest, StressHdfsTest, ScriptEngineTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, TestRandomFaceting, ShardRoutingCustomTest, TestDistributedGrouping, TestFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, TestGroupingSearch, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestTrie, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestValueSourceCache, TermVectorComponentTest, RequiredFieldsTest, TestArbitraryIndexDir, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, ReturnFieldsTest, JSONWriterTest, TestCSVResponseWriter, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, TestSolrIndexConfig, SolrIndexConfigTest, TestStressRecovery, MultiTermTest, TestBinaryField, TestSearchPerf, NumericFieldsTest, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestCharFilters, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestFastWriter, TestFastOutputStream, URLClassifyProcessorTest, DateMathParserTest, PrimUtilsTest, SpellingQueryConverterTest, ClusterStateTest, TestLRUCache, TestDocumentBuilder, SystemInfoHandlerTest, FileUtilsTest, DistributedMLTComponentTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, ActionThrottleTest, AsyncCallRequestStatusResponseTest, CleanupOldIndexTest, CollectionStateFormat2Test, DeleteInactiveReplicaTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnShardRestartTest]
>    [junit4] Completed [439/542] on J2 in 364.94s, 1 test, 1 error <<< FAILURES!
>
> [...truncated 365 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 1 error, 31 ignored (18 assumptions) [seed: 7F78F76DDF75FAD1]
>
> Total time: 198 minutes 25 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org



-- 
Regards,
Shalin Shekhar Mangar.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 993 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/993/

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11227 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/init-core-data-001
   [junit4]   2> 2804605 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2804608 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer start zk server on port:52462
   [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2804712 INFO  (zkCallback-2280-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a70d2b6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2804716 INFO  (zkCallback-2281-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5128e657 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2804718 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2804719 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2804720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2804729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2804730 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2804731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2804732 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2804739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2804740 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2805008 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1
   [junit4]   2> 2805010 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2805012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@665b505c{/,null,AVAILABLE}
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68a3874{HTTP/1.1}{127.0.0.1:56746}
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2807895ms
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=56746, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/control/data, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores}
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/'
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2805027 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2805028 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2805029 INFO  (zkCallback-2282-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39baa32 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/solr.xml
   [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
   [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1984045032
   [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/]
   [junit4]   2> 2805044 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib
   [junit4]   2> 2805044 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib).
   [junit4]   2> 2805051 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2805054 INFO  (zkCallback-2284-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2e9d74e9 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2805055 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2805056 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2805057 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51ed1947 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2805061 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2805063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2805065 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2805066 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2805068 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2805069 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2805070 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2805071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2805072 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56746_
   [junit4]   2> 2805073 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56746_
   [junit4]   2> 2805074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2805075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2805076 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000
   [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56746_
   [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2805078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=94731933327425540-127.0.0.1:56746_-n_0000000000) starting
   [junit4]   2> 2805080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2805085 INFO  (OverseerCollectionConfigSetProcessor-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2805085 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2805108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2805109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
   [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, coreNodeName=, dataDir=data/, name=collection1, config=solrconfig.xml, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1, shard=, transient=false, schema=schema.xml}
   [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/
   [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2805113 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 0
   [junit4]   2> 2805113 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"}
   [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2805116 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/'
   [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2806129 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2806133 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2806145 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2806170 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2806171 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2806177 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2806414 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2806439 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2806440 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2806448 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2806451 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@379f19b4
   [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data
   [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2806457 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index
   [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7cd8dcec
   [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
   [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2806458 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/
   [junit4]   2> 2806459 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2806463 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2806465 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806466 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806468 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806469 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806472 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/update/json,/admin/segments,/admin/mbeans,/admin/ping,/update/csv,/admin/plugins,/update,/admin/file,/replication,standard,/update/json/docs,/admin/threads,/admin/properties,/admin/luke,/admin/system,/get,/schema,/config
   [junit4]   2> 2806473 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=82.4736328125, floorSegmentMB=1.5283203125, forceMergeDeletesPctAllowed=4.112648727330555, segmentsPerTier=14.0, maxCFSSegmentSizeMB=1.6865234375, noCFSRatio=0.0
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@772028e6[collection1] main
   [junit4]   2> 2806477 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2806479 INFO  (searcherExecutor-5949-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@772028e6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2806480 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515710935199645696
   [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515710935199645696
   [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2806482 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:56746 collection:control_collection shard:shard1
   [junit4]   2> 2806483 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2806486 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94731933327425540-core_node1-n_0000000000
   [junit4]   2> 2806487 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56746/collection1/
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2806488 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56746/collection1/ has no replicas
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56746/collection1/ shard1
   [junit4]   2> 2806489 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2806491 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2806493 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2806595 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806614 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806615 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@794415fc name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806618 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806619 INFO  (zkCallback-2288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77c64dbb name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:56746/collection1/ and leader is http://127.0.0.1:56746/collection1/
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56746
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 2
   [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"}
   [junit4]   2> 2806647 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2806750 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806750 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1
   [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001
   [junit4]   2> 2806892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@719f3138{/,null,AVAILABLE}
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3bc61657{HTTP/1.1}{127.0.0.1:50535}
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2809776ms
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/jetty1, hostContext=/, solrconfig=solrconfig.xml, hostPort=50535}
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/'
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2806909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806911 INFO  (zkCallback-2289-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@306792d6 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/solr.xml
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1559122149
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/]
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib
   [junit4]   2> 2806926 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib).
   [junit4]   2> 2806934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806937 INFO  (zkCallback-2291-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c57b1e6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806940 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806941 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@124679be name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806941 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806945 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50535_
   [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50535_
   [junit4]   2> 2807952 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2807952 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2807952 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2807954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425545-127.0.0.1:50535_-n_0000000001
   [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
   [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1, name=collection1, configSetProperties=configsetprops.json, shard=, coreNodeName=, loadOnStartup=true, dataDir=data/, schema=schema.xml, transient=false, config=solrconfig.xml}
   [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/
   [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50535",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:50535_"} current state version: 3
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50535",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:50535_"}
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2807976 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 2807977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2808078 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/'
   [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2808992 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2808997 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2809010 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2809039 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2809040 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2809046 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2809381 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2809388 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2809390 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2809400 INFO  (coreLoadExecutor-5959-thread-

[...truncated too long message...]

ill nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3169248 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [n:127.0.0.1:33453_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001
   [junit4]   2> Oct 22, 2015 2:22:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=be_BY, timezone=Pacific/Port_Moresby
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=204385304,total=520093696
   [junit4]   2> NOTE: All tests run in this JVM: [TestExceedMaxTermLength, BinaryUpdateRequestHandlerTest, TestAuthorizationFramework, TestNamedUpdateProcessors, CoreAdminCreateDiscoverTest, TestCloudManagedSchema, TestRemoveLastDynamicCopyField, TestRawResponseWriter, TestCloudSchemaless, ZkStateReaderTest, TestZkChroot, TestSolrQueryParser, TestReload, TestReversedWildcardFilterFactory, HttpPartitionTest, TestSchemaManager, TestPostingsSolrHighlighter, HdfsLockFactoryTest, QueryParsingTest, TestIndexingPerformance, TestCloudManagedSchemaConcurrent, RAMDirectoryFactoryTest, CloudExitableDirectoryReaderTest, CoreAdminHandlerTest, VMParamsZkACLAndCredentialsProvidersTest, SampleTest, AutoCommitTest, TestSolrDeletionPolicy1, StatsComponentTest, SolrCoreCheckLockOnStartupTest, AsyncMigrateRouteKeyTest, TestSimpleQParserPlugin, SimpleCollectionCreateDeleteTest, SuggesterWFSTTest, VersionInfoTest, ChangedSchemaMergeTest, TestPKIAuthenticationPlugin, TestIndexSearcher, DistributedQueryComponentCustomSortTest, TestMacros, PrimitiveFieldTypeTest, DeleteShardTest, PluginInfoTest, HdfsRecoveryZkTest, SSLMigrationTest, DistributedFacetPivotSmallAdvancedTest, RequestHandlersTest, BadComponentTest, TestSearcherReuse, DocValuesMultiTest, ConnectionManagerTest, FullSolrCloudDistribCmdsTest, AnalyticsMergeStrategyTest, DeleteLastCustomShardedReplicaTest, TestConfigSets, TestCustomSort, TestMaxScoreQueryParser, CollectionReloadTest, TestInitParams, ExitableDirectoryReaderTest, DistribCursorPagingTest, DocValuesMissingTest, DebugComponentTest, TestSystemIdResolver, AnalysisErrorHandlingTest, TestSortByMinMaxFunction, DirectUpdateHandlerOptimizeTest, TestFieldTypeCollectionResource, SearchHandlerTest, TestElisionMultitermQuery, TestManagedStopFilterFactory, TestJmxIntegration, RulesTest, HighlighterMaxOffsetTest, TestCollationField, LukeRequestHandlerTest, TestDocBasedVersionConstraints, TestDefaultSearchFieldResource, FieldAnalysisRequestHandlerTest, FieldMutatingUpdateProcessorTest, TestPivotHelperCode, DocExpirationUpdateProcessorFactoryTest, StressHdfsTest, ScriptEngineTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, TestRandomFaceting, ShardRoutingCustomTest, TestDistributedGrouping, TestFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, TestGroupingSearch, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestTrie, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestValueSourceCache, TermVectorComponentTest, RequiredFieldsTest, TestArbitraryIndexDir, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, ReturnFieldsTest, JSONWriterTest, TestCSVResponseWriter, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, TestSolrIndexConfig, SolrIndexConfigTest, TestStressRecovery, MultiTermTest, TestBinaryField, TestSearchPerf, NumericFieldsTest, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestCharFilters, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestFastWriter, TestFastOutputStream, URLClassifyProcessorTest, DateMathParserTest, PrimUtilsTest, SpellingQueryConverterTest, ClusterStateTest, TestLRUCache, TestDocumentBuilder, SystemInfoHandlerTest, FileUtilsTest, DistributedMLTComponentTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, ActionThrottleTest, AsyncCallRequestStatusResponseTest, CleanupOldIndexTest, CollectionStateFormat2Test, DeleteInactiveReplicaTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [439/542] on J2 in 364.94s, 1 test, 1 error <<< FAILURES!

[...truncated 365 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 1 error, 31 ignored (18 assumptions) [seed: 7F78F76DDF75FAD1]

Total time: 198 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 992 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/992/

2 tests failed.
FAILED:  org.apache.lucene.codecs.memory.TestDirectDocValuesFormat.testRandomSortedBytes

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.codecs.memory.TestDirectDocValuesFormat

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)




Build Log:
[...truncated 6374 lines...]
   [junit4] Suite: org.apache.lucene.codecs.memory.TestDirectDocValuesFormat
   [junit4]   2> Oct 21, 2015 2:34:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.codecs.memory.TestDirectDocValuesFormat
   [junit4]   2>    1) Thread[id=2756, name=SUITE-TestDirectDocValuesFormat-seed#[A183E624F885474E], state=RUNNABLE, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:676)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:587)
   [junit4]   2>    2) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:597)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:446)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2>    3) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:49)
   [junit4]   2>    4) Thread[id=2757, name=TEST-TestDirectDocValuesFormat.testRandomSortedBytes-seed#[A183E624F885474E], state=TIMED_WAITING, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4296)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1785)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1719)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.doRandomForceMerge(RandomIndexWriter.java:340)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:355)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:301)
   [junit4]   2>         at org.apache.lucene.index.BaseDocValuesFormatTestCase.testRandomSortedBytes(BaseDocValuesFormatTestCase.java:1146)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDirectDocValuesFormat -Dtests.method=testRandomSortedBytes -Dtests.seed=A183E624F885474E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_MT -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   7191s J0 | TestDirectDocValuesFormat.testRandomSortedBytes <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)
   [junit4]   2> Oct 21, 2015 2:34:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> Oct 21, 2015 2:34:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.memory.TestDirectDocValuesFormat: 
   [junit4]   2>    1) Thread[id=2757, name=TEST-TestDirectDocValuesFormat.testRandomSortedBytes-seed#[A183E624F885474E], state=TIMED_WAITING, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4296)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1785)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1719)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.doRandomForceMerge(RandomIndexWriter.java:340)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:355)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:301)
   [junit4]   2>         at org.apache.lucene.index.BaseDocValuesFormatTestCase.testRandomSortedBytes(BaseDocValuesFormatTestCase.java:1146)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Oct 21, 2015 2:34:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=2757, name=TEST-TestDirectDocValuesFormat.testRandomSortedBytes-seed#[A183E624F885474E], state=TIMED_WAITING, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/test/J0/temp/lucene.codecs.memory.TestDirectDocValuesFormat_A183E624F885474E-001
   [junit4]   2> Oct 20, 2015 8:34:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {id=DFR I(n)L3(800.0), docId=DFR I(F)L3(800.0)}, locale=en_MT, timezone=Europe/San_Marino
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=211788168,total=341311488
   [junit4]   2> NOTE: All tests run in this JVM: [TestVarGapFixedIntervalPostingsFormat, TestAutoPrefixPostingsFormat, TestSimpleTextNormsFormat, TestMemoryPostingsFormat, TestSimpleTextPostingsFormat, TestDirectDocValuesFormat]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDirectDocValuesFormat -Dtests.seed=A183E624F885474E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_MT -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | TestDirectDocValuesFormat (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)
   [junit4] Completed [21/21] on J0 in 7220.16s, 9 tests, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 21 suites, 616 tests, 1 suite-level error, 1 error, 1 ignored (1 assumption) [seed: A183E624F885474E]

Total time: 215 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 991 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/991/

1 tests failed.
FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([D2641310986A09D9:316048953371CD9E]:0)
	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
	at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
	at org.apache.lucene.util.fst.Builder.add(Builder.java:417)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:258)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.access$500(MemoryPostingsFormat.java:113)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:400)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:173)
	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)




Build Log:
[...truncated 1524 lines...]
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDuelingCodecsAtNight -Dtests.method=testBigEquals -Dtests.seed=D2641310986A09D9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=pt -Dtests.timezone=America/Santarem -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    466s J1 | TestDuelingCodecsAtNight.testBigEquals <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D2641310986A09D9:316048953371CD9E]:0)
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]    > 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.add(Builder.java:417)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:258)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.access$500(MemoryPostingsFormat.java:113)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:400)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]    > 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:173)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestDuelingCodecsAtNight_D2641310986A09D9-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=pt, timezone=America/Santarem
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=111720776,total=435159040
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldMaskingSpanQuery, TestPerFieldDocValuesFormat, TestIndexInput, TestMultiTermQueryRewrites, TestCachingTokenFilter, TestFSTs, TestDocument, TestTransactionRollback, TestIndexReaderClose, TestSortRescorer, TestMultiCollector, TestIntBlockPool, TestCrash, TestLockFactory, TestLucene50SegmentInfoFormat, TestUsageTrackingFilterCachingPolicy, TestDocumentsWriterStallControl, TestDocCount, TestIndexWriterOnDiskFull, TestUnicodeUtil, TestLazyProxSkipping, TestSegmentMerger, TestFixedBitSet, TestDateSort, TestCloseableThreadLocal, TestSpanOrQuery, TestMultiTermsEnum, TestWeakIdentityMap, TestBoostQuery, TestLucene50FieldInfoFormat, TestMergeSchedulerExternal, TestPackedInts, TestSmallFloat, TestDemoParallelLeafReader, TestTermRangeFilter, TestFilteredSearch, TestBooleanQueryVisitSubscorers, TestFieldValueQuery, TestFlushByRamOrCountsPolicy, TestFieldCacheRewriteMethod, MultiCollectorTest, TestFastDecompressionMode, TestCheckIndex, TestSpansAdvanced2, TestNewestSegment, TestScoreCachingWrappingScorer, TestMathUtil, TestTermRangeQuery, TestIndexWriterExceptions, TestSortedNumericSortField, TestIndexSearcher, TestFieldValueFilter, TestRecyclingByteBlockAllocator, TestFieldReuse, TestCodecs, TestExitableDirectoryReader, TestOmitPositions, TestBlockPostingsFormat3, TestExceedMaxTermLength, TestSearchAfter, TestSameScoresWithThreads, TestIntArrayDocIdSet, TestBoolean2, TestLogMergePolicy, TestStringHelper, TestAllFilesHaveCodecHeader, TestTermScorer, TestFilterDirectoryReader, TestFlex, TestLucene50StoredFieldsFormat, TestLucene50StoredFieldsFormatHighCompression, TestFieldType, Test2BSortedDocValuesFixedSorted, Test2BSortedDocValuesOrds, TestAllFilesHaveChecksumFooter, TestCodecUtil, TestDocInverterPerFieldErrorInfo, TestDocValues, TestDuelingCodecsAtNight]
   [junit4] Completed [359/411] on J1 in 487.97s, 3 tests, 1 error <<< FAILURES!

[...truncated 185 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J1-20151019_160522_549.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid6468.hprof ...
   [junit4] Heap dump file created [422116720 bytes in 0.619 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 411 suites (1 ignored), 3311 tests, 1 error, 30 ignored (26 assumptions) [seed: D2641310986A09D9]

Total time: 50 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 990 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/990/

2 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Captured an uncaught exception in thread: Thread[id=48950, name=Thread-48086, state=RUNNABLE, group=TGRP-TestDistributedSearch]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=48950, name=Thread-48086, state=RUNNABLE, group=TGRP-TestDistributedSearch]
	at __randomizedtesting.SeedInfo.seed([411FC918F9FFEC9E:C94BF6C257038166]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33250//collection1: java.lang.NullPointerException
	at org.apache.solr.search.grouping.distributed.responseprocessor.StoredFieldsShardResponseProcessor.process(StoredFieldsShardResponseProcessor.java:42)
	at org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:749)
	at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:732)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:409)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:667)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)

	at __randomizedtesting.SeedInfo.seed([411FC918F9FFEC9E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.TestDistributedSearch$1.run(TestDistributedSearch.java:1110)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=5249, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5249, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51918/bx/e: collection already exists: awholynewstresscollection_collection3_9
	at __randomizedtesting.SeedInfo.seed([411FC918F9FFEC9E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 10523 lines...]
   [junit4] Suite: org.apache.solr.TestDistributedSearch
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/init-core-data-001
   [junit4]   2> 312671 INFO  (SUITE-TestDistributedSearch-seed#[411FC918F9FFEC9E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 312671 INFO  (SUITE-TestDistributedSearch-seed#[411FC918F9FFEC9E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 312967 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1
   [junit4]   2> 312968 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 312970 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@151bcc34{/,null,AVAILABLE}
   [junit4]   2> 312973 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@21add2ac{HTTP/1.1}{127.0.0.1:45725}
   [junit4]   2> 312973 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.Server Started @317106ms
   [junit4]   2> 312973 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=45725, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores, hostContext=/}
   [junit4]   2> 312974 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 312974 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/'
   [junit4]   2> 312974 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 312974 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 312989 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/solr.xml
   [junit4]   2> 313015 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores
   [junit4]   2> 313015 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer New CoreContainer 943324577
   [junit4]   2> 313015 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/]
   [junit4]   2> 313015 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/lib
   [junit4]   2> 313016 WARN  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/lib).
   [junit4]   2> 313025 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 313026 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 313026 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 313026 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 313026 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 313027 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 313027 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores
   [junit4]   2> 313028 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1, config=solrconfig.xml, configSetProperties=configsetprops.json, dataDir=data/, name=collection1, loadOnStartup=true, configSet=collection1, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/, transient=false}
   [junit4]   2> 313028 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/
   [junit4]   2> 313028 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 313028 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 313030 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 313030 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 313030 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 313030 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 313031 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 313031 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 313031 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 313054 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 313074 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 313116 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 313116 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 313126 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 313369 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 313377 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 313379 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 313385 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 313387 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 313390 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 313390 WARN  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 313390 WARN  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 313390 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 313390 WARN  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 313390 WARN  (coreLoadExecutor-615-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 313391 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 313391 INFO  (coreLoadExecutor-615-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 313391 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 313392 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 313392 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d9a661b
   [junit4]   2> 313392 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/data
   [junit4]   2> 313392 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/data/index/
   [junit4]   2> 313392 WARN  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 313392 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/data/index
   [junit4]   2> 313393 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 313393 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@51a02d15 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6aefe790),segFN=segments_1,generation=1}
   [junit4]   2> 313393 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 313394 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/data/
   [junit4]   2> 313394 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/control/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 313400 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 313400 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 313400 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 313401 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 313401 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 313401 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 313401 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 313401 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 313401 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 313402 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 313402 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 313402 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 313404 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 313406 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 313407 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 313408 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 313414 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 313416 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: tvrh,/replication,/admin/ping,/admin/system,/config,spellCheckCompRH1,/admin/file,/update/json/docs,/admin/mbeans,/terms,/admin/threads,standard,/admin/luke,/admin/segments,/schema,/search-facet-def,lazy,/admin/plugins,spellCheckCompRH,/search-facet-invariants,mltrh,/update,spellCheckCompRH_Direct,dismax,spellCheckWithWordbreak,/update/json,/admin/logging,/update/csv,/debug/dump,/admin/properties,/mlt,/get,mock,spellCheckWithWordbreak_Direct,defaults
   [junit4]   2> 313417 WARN  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 313418 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 313419 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 313419 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 313419 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 313419 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 313420 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=71.8271484375, floorSegmentMB=1.0673828125, forceMergeDeletesPctAllowed=11.883640014090801, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8117931013982642
   [junit4]   2> 313420 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@51a02d15 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6aefe790),segFN=segments_1,generation=1}
   [junit4]   2> 313420 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 313420 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@79c496c2[collection1] main
   [junit4]   2> 313421 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 313421 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 313421 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 313421 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 313431 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 313463 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 313463 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 313464 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 313464 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 313464 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 313464 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 313464 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 313464 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 313464 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 313465 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 313465 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515417442105425920
   [junit4]   2> 313465 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 313465 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 313466 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 313467 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@79c496c2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 313467 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515417442105425920
   [junit4]   2> 313467 INFO  (coreLoadExecutor-615-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 313829 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1
   [junit4]   2> 313831 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 313832 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35e591a0{/,null,AVAILABLE}
   [junit4]   2> 313833 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@72bbbb04{HTTP/1.1}{127.0.0.1:42638}
   [junit4]   2> 313833 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.e.j.s.Server Started @317966ms
   [junit4]   2> 313833 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=42638, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores}
   [junit4]   2> 313834 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 313834 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/'
   [junit4]   2> 313834 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 313834 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 313850 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/solr.xml
   [junit4]   2> 313864 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores
   [junit4]   2> 313864 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer New CoreContainer 2090560577
   [junit4]   2> 313864 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/]
   [junit4]   2> 313864 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/lib
   [junit4]   2> 313865 WARN  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/lib).
   [junit4]   2> 313872 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 313874 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 313874 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 313874 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 313874 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 313874 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 313875 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores
   [junit4]   2> 313876 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, configSet=collection1, dataDir=data/, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1, schema=schema.xml, configSetProperties=configsetprops.json, loadOnStartup=true, transient=false}
   [junit4]   2> 313876 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/
   [junit4]   2> 313876 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 313876 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 313877 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 313877 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 313877 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 313877 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 313878 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 313878 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 313878 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 313901 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 313921 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 313963 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 313963 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 313973 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 314224 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 314232 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 314234 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 314250 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 314253 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 314255 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 314256 WARN  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 314256 WARN  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 314256 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 314256 WARN  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 314256 WARN  (coreLoadExecutor-626-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 314257 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 314257 INFO  (coreLoadExecutor-626-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 314257 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 314257 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 314257 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d9a661b
   [junit4]   2> 314258 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/data
   [junit4]   2> 314258 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/data/index/
   [junit4]   2> 314258 WARN  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 314258 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 314259 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 314259 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@17ed2101 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5111d4ec),segFN=segments_1,generation=1}
   [junit4]   2> 314259 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 314260 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/data/
   [junit4]   2> 314260 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_411FC918F9FFEC9E-001/tempDir-001/shard0/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 314268 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 314269 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 314269 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 314269 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 314272 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 314273 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 314275 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 314276 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 314284 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 314288 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: tvrh,/replication,/admin/ping,/admin/system,/config,spellCheckCompRH1,/admin/file,/update/json/docs,/admin/mbeans,/terms,/admin/threads,standard,/admin/luke,/admin/segments,/schema,/search-facet-def,lazy,/admin/plugins,spellCheckCompRH,/search-facet-invariants,mltrh,/update,spellCheckCompRH_Direct,dismax,spellCheckWithWordbreak,/update/json,/admin/logging,/update/csv,/debug/dump,/admin/properties,/mlt,/get,mock,spellCheckWithWordbreak_Direct,defaults
   [junit4]   2> 314289 WARN  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 314290 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 314290 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 314290 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 314291 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 314291 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 314292 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=71.8271484375, floorSegmentMB=1.0673828125, forceMergeDeletesPctAllowed=11.883640014090801, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8117931013982642
   [junit4]   2> 314292 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@17ed2101 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5111d4ec),segFN=segments_1,generation=1}
   [junit4]   2> 314292 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 314292 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@39836bf4[collection1] main
   [junit4]   2> 314293 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 314293 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 314293 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 314293 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 314300 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 314338 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 314338 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 314339 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 314339 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 314339 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 314339 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 314340 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 314340 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 314340 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 314340 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 314340 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 314340 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515417443022929920
   [junit4]   2> 314341 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 314341 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 314342 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@39836bf4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 314343 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515417443022929920
   [junit4]   2> 314343 INFO  (coreLoadExecutor-626-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 314378 INFO  (TEST-TestDistributedSearch.test-seed#[411FC918F9FFEC9E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 314385 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1515417443065921536)} 0 4
   [junit4]   2> 314389 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1515417443072212992)} 0 1
   [junit4]   2> 314396 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1515417443078504448)]} 0 3
   [junit4]   2> 314401 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1515417443084795904)]} 0 2
   [junit4]   2> 314405 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1515417443090038784)]} 0 1
   [junit4]   2> 314409 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1515417443093184512)]} 0 1
   [junit4]   2> 314412 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1515417443097378816)]} 0 1
   [junit4]   2> 314416 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1515417443101573120)]} 0 1
   [junit4]   2> 314420 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1515417443105767424)]} 0 1
   [junit4]   2> 314423 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1515417443108913152)]} 0 1
   [junit4]   2> 314427 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1515417443113107456)]} 0 1
   [junit4]   2> 314431 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1515417443117301760)]} 0 1
   [junit4]   2> 314434 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1515417443120447488)]} 0 1
   [junit4]   2> 314437 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1515417443124641792)]} 0 1
   [junit4]   2> 314441 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1515417443127787520)]} 0 1
   [junit4]   2> 314444 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1515417443130933248)]} 0 1
   [junit4]   2> 314448 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1515417443135127552)]} 0 1
   [junit4]   2> 314451 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1515417443138273280)]} 0 1
   [junit4]   2> 314454 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1515417443141419008)]} 0 1
   [junit4]   2> 314457 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1515417443145613312)]} 0 1
   [junit4]   2> 314460 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 314465 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@51a02d15 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6aefe790),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@51a02d15 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6aefe790),segFN=segments_2,generation=2}
   [junit4]   2> 314465 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 314467 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1f9f93d4[collection1] main
   [junit4]   2> 314467 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 314469 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1f9f93d4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):c9)))}
   [junit4]   2> 314469 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2} {commit=} 0 9
   [junit4]   2> 314472 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 314477 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@17ed2101 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5111d4ec),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@17ed2101 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5111d4ec),segFN=segments_2,generation=2}
   [junit4]   2> 314477 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 314479 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@441527d0[collection1] main
   [junit4]   2> 314479 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 314480 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@441527d0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):c9)))}
   [junit4]   2> 314481 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2} {commit=} 0 9
   [junit4]   2> 314486 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1515417443173924864)]} 0 2
   [junit4]   2> 314491 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1515417443179167744)]} 0 2
   [junit4]   2> 314495 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1515417443183362048)]} 0 1
   [junit4]   2> 314498 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1515417443187556352)]} 0 1
   [junit4]   2> 314502 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1515417443191750656)]} 0 1
   [junit4]   2> 314505 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1515417443194896384)]} 0 1
   [junit4]   2> 314508 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1515417443199090688)]} 0 1
   [junit4]   2> 314512 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1515417443202236416)]} 0 1
   [junit4]   2> 314515 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1001 (1515417443205382144)]} 0 1
   [junit4]   2> 314518 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1001 (1515417443209576448)]} 0 1
   [junit4]   2> 314522 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1515417443212722176)]} 0 1
   [junit4]   2> 314525 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1515417443215867904)]} 0 0
   [junit4]   2> 314528 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1515417443220062208)]} 0 0
   [junit4]   2> 314531 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1515417443223207936)]} 0 0
   [junit4]   2> 314535 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1515417443226353664)]} 0 0
   [junit4]   2> 314538 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1515417443229499392)]} 0 0
   [junit4]   2> 314542 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1515417443234742272)]} 0 2
   [junit4]   2> 314547 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1515417443238936576)]} 0 2
   [junit4]   2> 314550 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[100 (1515417443242082304)]} 0 0
   [junit4]   2> 314553 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[100 (1515417443245228032)]} 0 0
   [junit4]   2> 314556 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[101 (1515417443249422336)]} 0 0
   [junit4]   2> 314559 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[101 (1515417443251519488)]} 0 0
   [junit4]   2> 314562 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[102 (1515417443255713792)]} 0 0
   [junit4]   2> 314565 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[102 (1515417443257810944)]} 0 0
   [junit4]   2> 314568 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[103 (1515417443260956672)]} 0 0
   [junit4]   2> 314571 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[103 (1515417443264102400)]} 0 0
   [junit4]   2> 314574 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[104 (1515417443268296704)]} 0 0
   [junit4]   2> 314577 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[104 (1515417443271442432)]} 0 0
   [junit4]   2> 314580 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[105 (1515417443274588160)]} 0 0
   [junit4]   2> 314583 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[105 (1515417443277733888)]} 0 0
   [junit4]   2> 314586 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[106 (1515417443280879616)]} 0 0
   [junit4]   2> 314589 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[106 (1515417443282976768)]} 0 0
   [junit4]   2> 314592 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[107 (1515417443287171072)]} 0 0
   [junit4]   2> 314595 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[107 (1515417443289268224)]} 0 0
   [junit4]   2> 314598 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[108 (1515417443292413952)]} 0 0
   [junit4]   2> 314601 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[108 (1515417443295559680)]} 0 0
   [junit4]   2> 314604 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[109 (1515417443298705408)]} 0 0
   [junit4]   2> 314607 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[109 (1515417443301851136)]} 0 0
   [junit4]   2> 314610 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[110 (1515417443304996864)]} 0 0
   [junit4]   2> 314613 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[110 (1515417443308142592)]} 0 0
   [junit4]   2> 314615 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[111 (1515417443311288320)]} 0 0
   [junit4]   2> 314618 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[111 (1515417443314434048)]} 0 0
   [junit4]   2> 314622 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[112 (1515417443317579776)]} 0 0
   [junit4]   2> 314625 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[112 (1515417443320725504)]} 0 0
   [junit4]   2> 314628 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[113 (1515417443323871232)]} 0 0
   [junit4]   2> 314631 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[113 (1515417443327016960)]} 0 0
   [junit4]   2> 314634 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[114 (1515417443331211264)]} 0 0
   [junit4]   2> 314637 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[114 (1515417443334356992)]} 0 0
   [junit4]   2> 314640 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[115 (1515417443337502720)]} 0 0
   [junit4]   2> 314643 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[115 (1515417443340648448)]} 0 0
   [junit4]   2> 314646 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[116 (1515417443343794176)]} 0 0
   [junit4]   2> 314649 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[116 (1515417443346939904)]} 0 0
   [junit4]   2> 314652 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[117 (1515417443350085632)]} 0 0
   [junit4]   2> 314655 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[117 (1515417443353231360)]} 0 0
   [junit4]   2> 314658 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[118 (1515417443356377088)]} 0 0
   [junit4]   2> 314661 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[118 (1515417443358474240)]} 0 0
   [junit4]   2> 314664 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[119 (1515417443361619968)]} 0 0
   [junit4]   2> 314667 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[119 (1515417443364765696)]} 0 0
   [junit4]   2> 314670 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[120 (1515417443367911424)]} 0 0
   [junit4]   2> 314672 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[120 (1515417443371057152)]} 0 0
   [junit4]   2> 314675 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[121 (1515417443374202880)]} 0 0
   [junit4]   2> 314678 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[121 (1515417443377348608)]} 0 0
   [junit4]   2> 314681 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[122 (1515417443380494336)]} 0 0
   [junit4]   2> 314684 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[122 (1515417443383640064)]} 0 0
   [junit4]   2> 314687 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[123 (1515417443386785792)]} 0 0
   [junit4]   2> 314690 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[123 (1515417443389931520)]} 0 0
   [junit4]   2> 314693 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[124 (1515417443393077248)]} 0 0
   [junit4]   2> 314696 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[124 (1515417443396222976)]} 0 0
   [junit4]   2> 314699 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[125 (1515417443399368704)]} 0 0
   [junit4]   2> 314702 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[125 (1515417443402514432)]} 0 0
   [junit4]   2> 314705 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[126 (1515417443405660160)]} 0 0
   [junit4]   2> 314708 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[126 (1515417443407757312)]} 0 0
   [junit4]   2> 314711 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[127 (1515417443411951616)]} 0 0
   [junit4]   2> 314714 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[127 (1515417443414048768)]} 0 0
   [junit4]   2> 314717 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[128 (1515417443417194496)]} 0 0
   [junit4]   2> 314720 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[128 (1515417443420340224)]} 0 0
   [junit4]   2> 314723 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[129 (1515417443423485952)]} 0 0
   [junit4]   2> 314726 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[129 (1515417443426631680)]} 0 0
   [junit4]   2> 314729 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[130 (1515417443429777408)]} 0 0
   [junit4]   2> 314732 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[130 (1515417443432923136)]} 0 0
   [junit4]   2> 314735 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[131 (1515417443436068864)]} 0 0
   [junit4]   2> 314738 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[131 (1515417443439214592)]} 0 0
   [junit4]   2> 314741 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[132 (1515417443442360320)]} 0 0
   [junit4]   2> 314744 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[132 (1515417443445506048)]} 0 0
   [junit4]   2> 314747 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[133 (1515417443448651776)]} 0 0
   [junit4]   2> 314750 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[133 (1515417443451797504)]} 0 0
   [junit4]   2> 314753 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[134 (1515417443454943232)]} 0 0
   [junit4]   2> 314756 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[134 (1515417443458088960)]} 0 0
   [junit4]   2> 314759 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[135 (1515417443461234688)]} 0 0
   [junit4]   2> 314762 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[135 (1515417443464380416)]} 0 0
   [junit4]   2> 314765 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[136 (1515417443467526144)]} 0 0
   [junit4]   2> 314768 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[136 (1515417443470671872)]} 0 0
   [junit4]   2> 314774 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[137 (1515417443476963328)]} 0 0
   [junit4]   2> 314777 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[137 (1515417443480109056)]} 0 0
   [junit4]   2> 314780 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[138 (1515417443483254784)]} 0 0
   [junit4]   2> 314782 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[138 (1515417443486400512)]} 0 0
   [junit4]   2> 314786 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[139 (1515417443489546240)]} 0 0
   [junit4]   2> 314788 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[139 (1515417443492691968)]} 0 0
   [junit4]   2> 314791 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[140 (1515417443495837696)]} 0 0
   [junit4]   2> 314794 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[140 (1515417443498983424)]} 0 0
   [junit4]   2> 314797 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[141 (1515417443502129152)]} 0 0
   [junit4]   2> 314800 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[141 (1515417443505274880)]} 0 0
   [junit4]   2> 314803 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[142 (1515417443507372032)]} 0 0
   [junit4]   2> 314806 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[142 (1515417443510517760)]} 0 0
   [junit4]   2> 314809 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[143 (1515417443513663488)]} 0 0
   [junit4]   2> 314812 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[143 (1515417443516809216)]} 0 0
   [junit4]   2> 314815 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[144 (1515417443519954944)]} 0 0
   [junit4]   2> 314818 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[144 (1515417443523100672)]} 0 0
   [junit4]   2> 314821 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[145 (1515417443526246400)]} 0 0
   [junit4]   2> 314824 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[145 (1515417443529392128)]} 0 0
   [junit4]   2> 314827 INFO  (qtp225098380-1219) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[146 (1515417443532537856)]} 0 0
   [junit4]   2> 314830 INFO  (qtp1522116154-1237) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[146 (1515417443535683584)]} 0 0
   [junit4]   2> 314833 INFO  (qtp225098380-1220) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[147 (1515417443538829312)]} 0 0
   [junit4]   2> 314836 INFO  (qtp1522116154-1238) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[147 (1515417443541975040)]} 0 0
   [junit4]   2> 314839 INFO  (qtp225098380-1221) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[148 (1515417443545120768)]} 0 0
   [junit4]   2> 314842 INFO  (qtp1522116154-1239) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[148 (1515417443548266496)]} 0 0
   [junit4]   2> 314845 INFO  (qtp225098380-1222) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[149 (1515417443551412224)]} 0 0
   [junit4]   2> 314847 INFO  (qtp1522116154-1240) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[149 (1515417443554557952)]} 0 0
   [junit4]   2> 314850 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 314894 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@51a02d15 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6aefe790),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@51a02d15 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6aefe790),segFN=segments_3,generation=3}
   [junit4]   2> 314894 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 314894 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@521d3a85[collection1] main
   [junit4]   2> 314895 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 314896 INFO  (searcherExecutor-616-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@521d3a85[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):c9) Uninverting(_1(5.4.0):c59)))}
   [junit4]   2> 314897 INFO  (qtp225098380-1218) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2} {commit=} 0 46
   [junit4]   2> 314899 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 314922 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@17ed2101 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5111d4ec),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@17ed2101 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5111d4ec),segFN=segments_3,generation=3}
   [junit4]   2> 314922 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 314922 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3579a635[collection1] main
   [junit4]   2> 314923 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 314924 INFO  (searcherExecutor-627-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3579a635[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):c9) Uninverting(_1(5.4.0):c59)))}
   [junit4]   2> 314924 INFO  (qtp1522116154-1236) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&versi

[...truncated too long message...]

-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_8/leader_elect/shard4/election/94713609056092173-core_node19-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_1/leader_elect/shard3/election/94713609056092176-core_node13-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_7/leader_elect/shard2/election/94713609056092166-core_node7-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_9/leader_elect/shard3/election/94713609056092166-core_node23-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_3/leader_elect/shard3/election/94713609056092170-core_node8-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_4/leader_elect/shard2/election/94713609056092176-core_node11-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_3/leader_elect/shard2/election/94713609056092179-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_4/leader_elect/shard2/election/94713609056092173-core_node2-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_8/leader_elect/shard3/election/94713609056092176-core_node2-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_9/leader_elect/shard3/election/94713609056092173-core_node20-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_4/leader_elect/shard1/election/94713609056092176-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_1/leader_elect/shard1/election/94713609056092179-core_node18-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_1/leader_elect/shard2/election/94713609056092166-core_node10-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_3/leader_elect/shard3/election/94713609056092176-core_node7-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_7/leader_elect/shard2/election/94713609056092170-core_node1-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_1/leader_elect/shard7/election/94713609056092166-core_node5-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	66	/solr/collections
   [junit4]   2> 	65	/solr/live_nodes
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=411FC918F9FFEC9E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_GB -Dtests.timezone=Africa/Windhoek -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    514s J1 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5249, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51918/bx/e: collection already exists: awholynewstresscollection_collection3_9
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([411FC918F9FFEC9E]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5246, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51918/bx/e: Could not find collection : awholynewstresscollection_collection0_7
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([411FC918F9FFEC9E]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]   2> 1117444 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[411FC918F9FFEC9E]-worker) [n:127.0.0.1:50029_bx%2Fe c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=en_GB, timezone=Africa/Windhoek
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=169402952,total=491782144
   [junit4]   2> NOTE: All tests run in this JVM: [TestHashQParserPlugin, OutOfBoxZkACLAndCredentialsProvidersTest, TestTrieFacet, TestSchemaVersionResource, SimpleMLTQParserTest, TestClusterStateMutator, ShardRoutingTest, TestWordDelimiterFilterFactory, TestInfoStreamLogging, SpellCheckCollatorTest, DateRangeFieldTest, BasicFunctionalityTest, TestRandomRequestDistribution, TestCrossCoreJoin, DocValuesTest, TestDFRSimilarityFactory, WordBreakSolrSpellCheckerTest, DistributedFacetPivotSmallTest, TestSurroundQueryParser, AssignTest, BlockCacheTest, TestRTGBase, ExplicitHLLTest, MinimalSchemaTest, RecoveryAfterSoftCommitTest, SpatialFilterTest, TestFastOutputStream, TestLMJelinekMercerSimilarityFactory, PreAnalyzedFieldTest, BigEndianAscendingWordSerializerTest, SimplePostToolTest, TestLRUStatsCache, TestQueryTypes, NotRequiredUniqueKeyTest, TestConfigOverlay, TestHashPartitioner, TestRestoreCore, DirectSolrSpellCheckerTest, TestMiniSolrCloudCluster, InfoHandlerTest, MergeStrategyTest, TriLevelCompositeIdRoutingTest, TestComplexPhraseQParserPlugin, TestScoreJoinQPNoScore, TestCloudPivotFacet, OverseerTaskQueueTest, TestSolrConfigHandlerConcurrent, ActionThrottleTest, ConcurrentDeleteAndCreateCollectionTest, TestQuerySenderListener, SolrIndexConfigTest, TestScoreJoinQPScore, CollectionsAPIDistributedZkTest]
   [junit4] Completed [186/542] on J1 in 514.88s, 1 test, 1 error <<< FAILURES!

[...truncated 1169 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 2 errors, 31 ignored (18 assumptions) [seed: 411FC918F9FFEC9E]

Total time: 288 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 989 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/989/

4 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.test

Error Message:
Server refused connection at: http://127.0.0.1:33348/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:33348/collection1
	at __randomizedtesting.SeedInfo.seed([B131DFC64F4A0680:3965E01CE1B66B78]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:582)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:365)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:98)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 48 more


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=11056, name=coreZkRegister-3366-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11056, name=coreZkRegister-3366-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([B131DFC64F4A0680]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=9234, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9234, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35179/y/b: Could not find collection : awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([B131DFC64F4A0680]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=54641, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=54641, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([B131DFC64F4A0680:3965E01CE1B66B78]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37684/lbm/sc: Could not find collection : awholynewstresscollection_collection1_4
	at __randomizedtesting.SeedInfo.seed([B131DFC64F4A0680]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10398 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/init-core-data-001
   [junit4]   2> 672623 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B131DFC64F4A0680]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 672623 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B131DFC64F4A0680]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /y/b
   [junit4]   2> 672630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 672637 INFO  (Thread-6330) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 672637 INFO  (Thread-6330) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 672737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.ZkTestServer start zk server on port:41377
   [junit4]   2> 672737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 672757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 672767 INFO  (zkCallback-451-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5fbf5051 name:ZooKeeperConnection Watcher:127.0.0.1:41377 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 672768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 672768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 672770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 672773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 672773 INFO  (zkCallback-452-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f9197b2 name:ZooKeeperConnection Watcher:127.0.0.1:41377/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 672774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 672774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 672775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 672776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 672778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 672779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 672779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 672781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 672781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 672783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 672783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 672785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 672785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 672786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 672786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 672787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 672788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 672789 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 672789 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 672791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 672791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 672792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 672793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 672794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 672794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 672796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 672796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 673116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1
   [junit4]   2> 673118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 673130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47f6823b{/y/b,null,AVAILABLE}
   [junit4]   2> 673133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2f5b3917{HTTP/1.1}{127.0.0.1:34057}
   [junit4]   2> 673133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.Server Started @675960ms
   [junit4]   2> 673134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/tempDir-001/control/data, hostContext=/y/b, hostPort=34057}
   [junit4]   2> 673134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 673134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/'
   [junit4]   2> 673134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 673134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 673149 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 673151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 673151 INFO  (zkCallback-453-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7321a7bd name:ZooKeeperConnection Watcher:127.0.0.1:41377/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 673151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 673152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 673153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 673153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/solr.xml
   [junit4]   2> 673167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores
   [junit4]   2> 673168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoreContainer New CoreContainer 1703385959
   [junit4]   2> 673168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/]
   [junit4]   2> 673168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/lib
   [junit4]   2> 673168 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/lib).
   [junit4]   2> 673175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 673177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 673177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 673177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 673177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41377/solr
   [junit4]   2> 673177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 673177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 673179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 673179 INFO  (zkCallback-455-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27c5a6d5 name:ZooKeeperConnection Watcher:127.0.0.1:41377 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 673180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 673180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 673182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 673183 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b4b4120 name:ZooKeeperConnection Watcher:127.0.0.1:41377/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 673183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 673184 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 673185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 673188 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 673189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 673191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 673193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 673194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 673195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 673196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 673197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34057_y%2Fb
   [junit4]   2> 673197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34057_y%2Fb
   [junit4]   2> 673198 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 673199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 673200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 673201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000
   [junit4]   2> 673201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34057_y%2Fb
   [junit4]   2> 673201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 673202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer Overseer (id=94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) starting
   [junit4]   2> 673204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 673208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 673209 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 673210 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 673212 INFO  (OverseerCollectionConfigSetProcessor-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 673215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 673228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 673228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 673229 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores
   [junit4]   2> 673229 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, shard=, configSetProperties=configsetprops.json, collection=control_collection, name=collection1, coreNodeName=, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1, loadOnStartup=true, schema=schema.xml}
   [junit4]   2> 673230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/
   [junit4]   2> 673230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 673231 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 673231 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 673231 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 673231 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 673233 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 673233 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34057/y/b",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:34057_y%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 673234 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34057/y/b",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:34057_y%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null}
   [junit4]   2> 673234 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 673234 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 673236 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 673236 INFO  (zkCallback-456-thread-2-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674234 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 674234 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 674234 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 674234 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 674235 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 674235 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/'
   [junit4]   2> 674235 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 674235 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 674265 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 674272 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 674301 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 674354 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 674355 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 674361 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 674745 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 674759 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 674761 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 674779 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 674782 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 674792 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 674793 WARN  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 674793 WARN  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 674793 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 674793 WARN  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 674793 WARN  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 674794 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 674795 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 674795 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 674795 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 674795 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a130ef5
   [junit4]   2> 674795 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/data
   [junit4]   2> 674801 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/data/index/
   [junit4]   2> 674802 WARN  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 674802 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/data/index
   [junit4]   2> 674802 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@a758507
   [junit4]   2> 674803 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@297093ee lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e000383),segFN=segments_1,generation=1}
   [junit4]   2> 674803 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 674813 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 674813 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 674813 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 674813 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 674814 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 674814 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 674814 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 674814 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 674814 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 674817 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 674818 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 674819 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 674821 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 674821 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/data/
   [junit4]   2> 674821 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 674829 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/admin/threads,/admin/mbeans,/update/json,/admin/properties,/update/json/docs,/admin/logging,/config,/admin/plugins,/admin/luke,/admin/system,/schema,standard,/replication,/admin/segments,/get,/admin/file,/update,/admin/ping
   [junit4]   2> 674830 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 674839 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 674840 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 674840 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 674840 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 674841 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=89.8466796875, floorSegmentMB=1.2373046875, forceMergeDeletesPctAllowed=1.427643221818864, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7125469316806492
   [junit4]   2> 674841 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@297093ee lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e000383),segFN=segments_1,generation=1}
   [junit4]   2> 674841 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 674841 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@33e881f8[collection1] main
   [junit4]   2> 674841 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 674870 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 674870 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 674870 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 674871 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 674871 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 674871 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 674871 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 674871 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 674871 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 674872 INFO  (searcherExecutor-1748-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33e881f8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 674872 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 674872 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 674872 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 674872 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515322912323665920
   [junit4]   2> 674874 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515322912323665920
   [junit4]   2> 674874 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 674875 INFO  (coreLoadExecutor-1747-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 674876 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34057/y/b collection:control_collection shard:shard1
   [junit4]   2> 674877 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 674880 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94707681874149380-core_node1-n_0000000000
   [junit4]   2> 674881 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 674883 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 674883 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 674883 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34057/y/b/collection1/
   [junit4]   2> 674883 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 674883 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34057/y/b/collection1/ has no replicas
   [junit4]   2> 674883 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 674883 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34057/y/b/collection1/ shard1
   [junit4]   2> 674884 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 674885 INFO  (zkCallback-456-thread-2-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674885 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674889 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 674892 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:34057/y/b",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 674994 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674994 INFO  (zkCallback-456-thread-2-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 675044 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:34057/y/b/collection1/ and leader is http://127.0.0.1:34057/y/b/collection1/
   [junit4]   2> 675044 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34057/y/b
   [junit4]   2> 675044 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 675045 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 675045 INFO  (coreZkRegister-1741-thread-1-processing-r:core_node1 n:127.0.0.1:34057_y%2Fb c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34057_y%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 675047 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34057/y/b",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:34057_y%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1"} current state version: 3
   [junit4]   2> 675047 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34057/y/b",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:34057_y%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 675255 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 675255 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 675256 INFO  (zkCallback-456-thread-2-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 675258 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 675259 INFO  (zkCallback-458-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@594fd0f5 name:ZooKeeperConnection Watcher:127.0.0.1:41377/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675259 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 675259 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 675259 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 675261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 675261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 675261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 675262 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 675263 INFO  (zkCallback-459-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3eab5c46 name:ZooKeeperConnection Watcher:127.0.0.1:41377/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675263 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 675263 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 675265 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 675265 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 675371 INFO  (OverseerStateUpdate-94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 675706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/cores/collection1
   [junit4]   2> 675707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001
   [junit4]   2> 675707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 675709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a9cae73{/y/b,null,AVAILABLE}
   [junit4]   2> 675710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.ServerConnector Started ServerConnector@34810576{HTTP/1.1}{127.0.0.1:40340}
   [junit4]   2> 675710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.e.j.s.Server Started @678537ms
   [junit4]   2> 675710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/tempDir-001/jetty1, hostPort=40340, hostContext=/y/b, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/cores}
   [junit4]   2> 675710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 675710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/'
   [junit4]   2> 675711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 675711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 675726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 675727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 675728 INFO  (zkCallback-460-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40d6ed8a name:ZooKeeperConnection Watcher:127.0.0.1:41377/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 675728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 675729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 675729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/solr.xml
   [junit4]   2> 675743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/cores
   [junit4]   2> 675743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoreContainer New CoreContainer 926989756
   [junit4]   2> 675743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/]
   [junit4]   2> 675743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/lib
   [junit4]   2> 675744 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/lib).
   [junit4]   2> 675751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 675753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 675753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 675753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 675753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41377/solr
   [junit4]   2> 675753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 675753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 675754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 675755 INFO  (zkCallback-462-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40d3b47b name:ZooKeeperConnection Watcher:127.0.0.1:41377 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 675756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 675758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 675760 INFO  (zkCallback-463-thread-1-processing-n:127.0.0.1:40340_y%2Fb) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e4149c9 name:ZooKeeperConnection Watcher:127.0.0.1:41377/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 675765 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 676772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40340_y%2Fb
   [junit4]   2> 676773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40340_y%2Fb
   [junit4]   2> 676775 INFO  (zkCallback-458-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 676775 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 676775 INFO  (zkCallback-456-thread-2-processing-n:127.0.0.1:34057_y%2Fb) [n:127.0.0.1:34057_y%2Fb    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 676775 INFO  (zkCallback-463-thread-1-processing-n:127.0.0.1:40340_y%2Fb) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 676778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 676778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94707681874149385-127.0.0.1:40340_y%2Fb-n_0000000001
   [junit4]   2> 676779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94707681874149380-127.0.0.1:34057_y%2Fb-n_0000000000 to know if I could be the leader
   [junit4]   2> 676796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 676796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 676797 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/cores
   [junit4]   2> 676798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/cores/collection1, transient=false, configSetProperties=configsetprops.json, dataDir=data/, schema=schema.xml, loadOnStartup=true, name=collection1, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/cores/collection1/, shard=, collection=collection1}
   [junit4]   2> 676798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B131DFC64F4A0680]) [n:127.0.0.1:40340_y%2Fb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B131DFC64F4A0680-001/shard-1-001/cores/collection1/
   [junit4]   2> 676798 INFO  (TEST-Colle

[...truncated too long message...]

junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3577914 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B131DFC64F4A0680]-worker) [n:127.0.0.1:51541_lbm%2Fsc c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Oct 17, 2015 8:17:26 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), other_tl1=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval), intDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), _version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_i1=PostingsFormat(name=LuceneVarGapDocFreqInterval), text=Lucene50(blocksize=128), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= true), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=Memory doPackFST= true)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=zh_SG, timezone=America/Chihuahua
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=215378824,total=532152320
   [junit4]   2> NOTE: All tests run in this JVM: [SolrCloudExampleTest, TestRecoveryHdfs, DirectUpdateHandlerTest, TestExpandComponent, CloudExitableDirectoryReaderTest, SimpleCollectionCreateDeleteTest, ZkStateWriterTest, TestIntervalFaceting, TestJmxIntegration, TestCollationField, QueryResultKeyTest, TestSweetSpotSimilarityFactory, StressHdfsTest, RegexBoostProcessorTest, TestManagedSynonymFilterFactory, OverseerTest, TestSimpleTrackingShardHandler, TestSolrDeletionPolicy1, TestCSVResponseWriter, TestSortByMinMaxFunction, HdfsRecoveryZkTest, VMParamsZkACLAndCredentialsProvidersTest, DebugComponentTest, BadIndexSchemaTest, TestTolerantSearch, AutoCommitTest, TestRTimerTree, TestCloudSchemaless, TestHighFrequencyDictionaryFactory, RAMDirectoryFactoryTest, TestExceedMaxTermLength, AnalysisErrorHandlingTest, TestSchemaManager, SolrCoreCheckLockOnStartupTest, TestManagedStopFilterFactory, TestSearcherReuse, TestBulkSchemaAPI, TestSimpleQParserPlugin, FullSolrCloudDistribCmdsTest, PrimitiveFieldTypeTest, CollectionReloadTest, ZkNodePropsTest, TestFastWriter, SuggesterWFSTTest, TestDocBasedVersionConstraints, TestAuthorizationFramework, HttpPartitionTest, DistributedFacetPivotSmallAdvancedTest, TestCursorMarkWithoutUniqueKey, DirectUpdateHandlerOptimizeTest, DistributedExpandComponentTest, TestIndexSearcher, TestDynamicFieldCollectionResource, TestConfigSetsAPIExclusivity, VersionInfoTest, ExitableDirectoryReaderTest, AsyncMigrateRouteKeyTest, ScriptEngineTest, TestRequestStatusCollectionAPI, HighlighterConfigTest, TestFuzzyAnalyzedSuggestions, HdfsDirectoryTest, UUIDFieldTest, TestSolrIndexConfig, LeaderElectionTest, TestCloudManagedSchema, HighlighterMaxOffsetTest, SystemInfoHandlerTest, TestGroupingSearch, RequestHandlersTest, TestSchemaSimilarityResource, FieldMutatingUpdateProcessorTest, QueryParsingTest, DocValuesMissingTest, TestDefaultSimilarityFactory, CoreAdminHandlerTest, LukeRequestHandlerTest, TestExtendedDismaxParser, DistributedQueryComponentCustomSortTest, TestRecovery, DistributedDebugComponentTest, RecoveryZkTest, DistribCursorPagingTest, TestUniqueKeyFieldResource, TestNamedUpdateProcessors, TestSort, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, ShardRoutingTest, TestReplicationHandler, TestStressReorder, TestReload, HardAutoCommitTest, StatsComponentTest, TestAtomicUpdateErrorCases, StatelessScriptUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestOmitPositions, XmlUpdateRequestHandlerTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, JSONWriterTest, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, TestLFUCache, ConnectionManagerTest, CustomCollectionTest, DistributedQueueTest, LeaderInitiatedRecoveryOnCommitTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, SharedFSAutoReplicaFailoverUtilsTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCollectionAPI, TestLeaderInitiatedRecoveryThread, TestMiniSolrCloudClusterBase, TestMiniSolrCloudClusterSSL, TestShortCircuitedRequests, TestSolrCloudWithKerberosAlt, HdfsBasicDistributedZkTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [457/542] on J2 in 351.07s, 1 test, 1 error <<< FAILURES!

[...truncated 283 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 4 errors, 31 ignored (18 assumptions) [seed: B131DFC64F4A0680]

Total time: 195 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 988 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/988/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=11380, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11380, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43647/_jdw, http://127.0.0.1:41788/_jdw, http://127.0.0.1:57421/_jdw, http://127.0.0.1:48776/_jdw, http://127.0.0.1:50810/_jdw]
	at __randomizedtesting.SeedInfo.seed([5CB427BB0F96593A]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43647/_jdw, http://127.0.0.1:41788/_jdw, http://127.0.0.1:57421/_jdw, http://127.0.0.1:48776/_jdw, http://127.0.0.1:50810/_jdw]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50810/_jdw: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=24184, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=24184, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55949: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([5CB427BB0F96593A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10827 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/init-core-data-001
   [junit4]   2> 1245280 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[5CB427BB0F96593A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1245280 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[5CB427BB0F96593A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_jdw/
   [junit4]   2> 1245284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1245285 INFO  (Thread-5806) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1245285 INFO  (Thread-5806) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1245385 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.ZkTestServer start zk server on port:40348
   [junit4]   2> 1245385 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1245385 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1245392 INFO  (zkCallback-1098-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f422de8 name:ZooKeeperConnection Watcher:127.0.0.1:40348 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1245392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1245393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1245393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1245395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1245397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1245398 INFO  (zkCallback-1099-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@41683e52 name:ZooKeeperConnection Watcher:127.0.0.1:40348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1245398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1245398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1245398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1245402 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1245403 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1245404 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1245406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1245406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1245408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1245409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1245410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1245410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1245412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1245412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1245414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1245414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1245415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1245415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1245417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1245417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1245418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1245419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1245420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1245420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1245422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1245422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1245423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1245424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1245426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1245427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1245427 INFO  (zkCallback-1100-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6787bbab name:ZooKeeperConnection Watcher:127.0.0.1:40348 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1245428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1245428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1245428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1245429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1245430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1245431 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a93568c name:ZooKeeperConnection Watcher:127.0.0.1:40348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1245431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1245431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1245431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1245431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1245434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1245434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1245436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1245436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1245437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1245437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1245439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1245439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1245441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1245441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1245443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1245443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1245444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1245444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1245446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1245446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1245447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1245448 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1245449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1245449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1245451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1245451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1245726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1
   [junit4]   2> 1245727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1245733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@726c491a{/_jdw,null,AVAILABLE}
   [junit4]   2> 1245734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@233384a6{HTTP/1.1}{127.0.0.1:43647}
   [junit4]   2> 1245735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.Server Started @1248601ms
   [junit4]   2> 1245735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/tempDir-001/control/data, hostContext=/_jdw, hostPort=43647, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores}
   [junit4]   2> 1245736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1245736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/'
   [junit4]   2> 1245736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1245736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1245751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1245752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1245753 INFO  (zkCallback-1102-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@535497d7 name:ZooKeeperConnection Watcher:127.0.0.1:40348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1245753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1245754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1245754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1245755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/solr.xml
   [junit4]   2> 1245768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores
   [junit4]   2> 1245769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoreContainer New CoreContainer 605638120
   [junit4]   2> 1245769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/]
   [junit4]   2> 1245769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/lib
   [junit4]   2> 1245769 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/lib).
   [junit4]   2> 1245777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1245778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1245778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1245778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1245778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40348/solr
   [junit4]   2> 1245779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1245779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1245780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1245780 INFO  (zkCallback-1104-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@530eb1d6 name:ZooKeeperConnection Watcher:127.0.0.1:40348 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1245781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1245781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1245782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1245783 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@704cdd03 name:ZooKeeperConnection Watcher:127.0.0.1:40348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1245783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1245784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1245786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1245789 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1245791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1245793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1245794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1245796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1245797 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1245798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1245799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43647__jdw
   [junit4]   2> 1245800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43647__jdw
   [junit4]   2> 1245801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1245802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1245804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1245804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94702197786017798-127.0.0.1:43647__jdw-n_0000000000
   [junit4]   2> 1245804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43647__jdw
   [junit4]   2> 1245804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1245806 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer Overseer (id=94702197786017798-127.0.0.1:43647__jdw-n_0000000000) starting
   [junit4]   2> 1245807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1245813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1245813 INFO  (OverseerCollectionConfigSetProcessor-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1245813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1245814 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1245827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1245827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1245828 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores
   [junit4]   2> 1245829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, name=collection1, schema=schema.xml, configSetProperties=configsetprops.json, shard=, dataDir=data/, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1, coreNodeName=, transient=false, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/}
   [junit4]   2> 1245829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/
   [junit4]   2> 1245829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1245830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1245830 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1245830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:43647__jdw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1245830 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1245832 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43647/_jdw",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:43647__jdw"} current state version: 0
   [junit4]   2> 1245833 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43647/_jdw",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:43647__jdw"}
   [junit4]   2> 1245833 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1245833 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1245833 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1245835 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1246833 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1246834 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1246834 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1246834 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1246835 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1246835 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/'
   [junit4]   2> 1246835 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1246835 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1246849 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1246854 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1246867 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1246894 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1246895 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1246901 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1247154 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1247162 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1247163 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1247171 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1247174 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1247177 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1247177 WARN  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1247177 WARN  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1247177 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1247178 WARN  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1247178 WARN  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1247179 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1247179 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1247179 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1247179 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1247179 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@cf24c1a
   [junit4]   2> 1247180 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/data
   [junit4]   2> 1247180 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1247180 WARN  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1247180 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/data/index
   [junit4]   2> 1247181 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@49de8a01
   [junit4]   2> 1247181 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@31366d8a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6c519843),segFN=segments_1,generation=1}
   [junit4]   2> 1247181 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1247182 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/data/
   [junit4]   2> 1247182 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1247185 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1247185 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1247185 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1247185 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1247186 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1247186 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1247186 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1247186 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1247186 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1247189 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1247190 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1247191 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1247192 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1247196 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/properties,/schema,/replication,/admin/mbeans,/get,/admin/file,standard,/update,/update/json/docs,/update/json,/admin/ping,/config,/admin/threads,/update/csv,/admin/segments,/admin/luke,/admin/plugins,/admin/logging,/admin/system
   [junit4]   2> 1247197 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1247198 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1247198 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1247198 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1247198 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1247199 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=96.173828125, floorSegmentMB=2.0361328125, forceMergeDeletesPctAllowed=3.5044215493147624, segmentsPerTier=31.0, maxCFSSegmentSizeMB=1.921875, noCFSRatio=0.0
   [junit4]   2> 1247199 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@31366d8a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6c519843),segFN=segments_1,generation=1}
   [junit4]   2> 1247199 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1247199 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6aa64c55[collection1] main
   [junit4]   2> 1247199 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1247200 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1247200 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1247201 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1247201 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1247201 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1247201 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1247201 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1247201 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1247201 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1247202 INFO  (searcherExecutor-3251-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6aa64c55[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1247202 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1247202 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1247202 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1247202 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515235166585356288
   [junit4]   2> 1247204 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515235166585356288
   [junit4]   2> 1247204 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1247205 INFO  (coreLoadExecutor-3250-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1247205 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:43647/_jdw collection:control_collection shard:shard1
   [junit4]   2> 1247206 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1247209 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94702197786017798-core_node1-n_0000000000
   [junit4]   2> 1247209 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1247211 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1247211 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1247211 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43647/_jdw/collection1/
   [junit4]   2> 1247211 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1247211 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43647/_jdw/collection1/ has no replicas
   [junit4]   2> 1247211 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43647/_jdw/collection1/ shard1
   [junit4]   2> 1247211 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1247213 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1247215 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1247217 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43647/_jdw",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1247319 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1247332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1247333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1247334 INFO  (zkCallback-1107-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@489d4932 name:ZooKeeperConnection Watcher:127.0.0.1:40348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1247334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1247334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1247334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1247337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1247368 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:43647/_jdw/collection1/ and leader is http://127.0.0.1:43647/_jdw/collection1/
   [junit4]   2> 1247368 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43647/_jdw
   [junit4]   2> 1247368 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1247368 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1247368 INFO  (coreZkRegister-3244-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1247370 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43647/_jdw",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:43647__jdw"} current state version: 2
   [junit4]   2> 1247371 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43647/_jdw",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:43647__jdw"}
   [junit4]   2> 1247473 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1247473 INFO  (zkCallback-1107-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1247620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/cores/collection1
   [junit4]   2> 1247620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001
   [junit4]   2> 1247621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1247623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ac2181{/_jdw,null,AVAILABLE}
   [junit4]   2> 1247623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6937c705{HTTP/1.1}{127.0.0.1:50810}
   [junit4]   2> 1247623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.e.j.s.Server Started @1250489ms
   [junit4]   2> 1247623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_jdw, hostPort=50810, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 1247624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1247624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/'
   [junit4]   2> 1247624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1247624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1247639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1247640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1247641 INFO  (zkCallback-1108-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29999a42 name:ZooKeeperConnection Watcher:127.0.0.1:40348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1247641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1247641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1247642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1247642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/solr.xml
   [junit4]   2> 1247669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/cores
   [junit4]   2> 1247669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoreContainer New CoreContainer 224686968
   [junit4]   2> 1247669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/]
   [junit4]   2> 1247669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/lib
   [junit4]   2> 1247670 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/lib).
   [junit4]   2> 1247677 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1247678 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1247679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1247679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1247679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40348/solr
   [junit4]   2> 1247679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1247680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1247680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1247682 INFO  (zkCallback-1110-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7457164e name:ZooKeeperConnection Watcher:127.0.0.1:40348 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1247682 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1247682 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1247684 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1247685 INFO  (zkCallback-1111-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6346543 name:ZooKeeperConnection Watcher:127.0.0.1:40348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1247685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1247691 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1248696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50810__jdw
   [junit4]   2> 1248697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50810__jdw
   [junit4]   2> 1248698 INFO  (zkCallback-1107-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1248698 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1248698 INFO  (zkCallback-1111-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1248699 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1248700 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94702197786017802-127.0.0.1:50810__jdw-n_0000000001
   [junit4]   2> 1248700 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94702197786017798-127.0.0.1:43647__jdw-n_0000000000 to know if I could be the leader
   [junit4]   2> 1248716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1248716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1248717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/cores
   [junit4]   2> 1248717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, name=collection1, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/cores/collection1, schema=schema.xml, coreNodeName=, config=solrconfig.xml, configSetProperties=configsetprops.json, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/cores/collection1/, dataDir=data/, transient=false}
   [junit4]   2> 1248717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_5CB427BB0F96593A-001/shard-1-001/cores/collection1/
   [junit4]   2> 1248717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1248718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1248718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5CB427BB0F96593A]) [n:127.0.0.1:50810__jdw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1248719 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1248719 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1248720 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:50810/_jdw",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:50810__jdw"} current state version: 3
   [junit4]   2> 1248720 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1248721 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:50810/_jdw",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:50810__jdw"}
   [junit4]   2> 1248721 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1248721 INFO  (OverseerStateUpdate-94702197786017798-127.0.0.1:43647__jdw-n_0000000000) [n:127.0.0.1:43647__jdw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1248823 INFO  (zkCallback-1111-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1248823 INFO  (zkCallback-1107-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1248823 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:43647__jdw) [n:127.0.0.1:43647__jdw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1249721 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1249721 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1249721 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1249721 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1249722 INFO  (coreLoadExecutor-3261-thread-1-processing-n:127.0.0.1:50810__jdw) [n:127.0.0.1:50810__jdw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=

[...truncated too long message...]

shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3605917 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[5CB427BB0F96593A]-worker) [n:127.0.0.1:41780_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Oct 16, 2015 8:53:40 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_ES, timezone=Africa/Ouagadougou
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=245033008,total=524288000
   [junit4]   2> NOTE: All tests run in this JVM: [TestClusterStateMutator, RecoveryAfterSoftCommitTest, TestCrossCoreJoin, TestMissingGroups, PreAnalyzedFieldTest, WordBreakSolrSpellCheckerTest, ConvertedLegacyTest, DistanceUnitsTest, DistributedTermsComponentTest, ZkCLITest, TestShortCircuitedRequests, TestQueryTypes, DeleteInactiveReplicaTest, TestConfigSetImmutable, DateRangeFieldTest, TestComplexPhraseQParserPlugin, TestLMJelinekMercerSimilarityFactory, TestObjectReleaseTracker, MinimalSchemaTest, TestJmxMonitoredMap, ConcurrentDeleteAndCreateCollectionTest, TestSolrConfigHandlerConcurrent, CircularListTest, SpellPossibilityIteratorTest, MoreLikeThisHandlerTest, CustomCollectionTest, ShardRoutingCustomTest, TestInfoStreamLogging, TestMiniSolrCloudCluster, FastVectorHighlighterTest, CollectionsAPIAsyncDistributedZkTest, TestRebalanceLeaders, TestFastOutputStream, NumericFieldsTest, TestQuerySenderListener, SolrInfoMBeanTest, SharedFSAutoReplicaFailoverUtilsTest, TestDistributedGrouping, TestRTGBase, SchemaVersionSpecificBehaviorTest, DistribDocExpirationUpdateProcessorTest, BigEndianAscendingWordSerializerTest, TestExactSharedStatsCache, BasicFunctionalityTest, ActionThrottleTest, TermVectorComponentTest, TestOrdValues, BlockDirectoryTest, ClusterStateUpdateTest, HdfsThreadLeakTest, ZkControllerTest, SparseHLLTest, SharedFSAutoReplicaFailoverTest, SpatialFilterTest, DistributedIntervalFacetingTest, TestCodecSupport, TestQueryUtils, CSVRequestHandlerTest, TestFiltering, NotRequiredUniqueKeyTest, DistributedFacetPivotLargeTest, MultiTermTest, TestSuggestSpellingConverter, TestStressVersions, LeaderInitiatedRecoveryOnShardRestartTest, TestCryptoKeys, TestRandomRequestDistribution, CollectionStateFormat2Test, TestBM25SimilarityFactory, MergeStrategyTest, DistributedFacetPivotSmallTest, SimpleMLTQParserTest, ShardRoutingTest, IndexBasedSpellCheckerTest, SaslZkACLProviderTest, SimplePostToolTest, TestHashQParserPlugin, TestChildDocTransformer, AnalyticsQueryTest, HdfsWriteToMultipleCollectionsTest, TestSchemaResource, OutOfBoxZkACLAndCredentialsProvidersTest, TestFileDictionaryLookup, TestTrackingShardHandlerFactory, TestConfigOverlay, BasicDistributedZk2Test, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, TestReplicationHandler, ZkSolrClientTest, TestRandomDVFaceting, AutoCommitTest, DistributedSpellCheckComponentTest, TestStressReorder, TestReload, SolrCoreTest, TestGroupingSearch, TestFunctionQuery, HighlighterTest, SoftAutoCommitTest, TestTrie, PolyFieldTest, SolrCoreCheckLockOnStartupTest, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, LukeRequestHandlerTest, DocumentBuilderTest, PathHierarchyTokenizerFactoryTest, FieldAnalysisRequestHandlerTest, LoggingHandlerTest, IndexSchemaTest, TestCollationField, JSONWriterTest, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, PingRequestHandlerTest, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, TestQuerySenderNoQuery, ResponseLogComponentTest, SolrIndexConfigTest, TestStressRecovery, TestSolrDeletionPolicy2, SampleTest, TestSearchPerf, ExternalFileFieldSortTest, TestSolrCoreProperties, DirectSolrConnectionTest, TestLuceneMatchVersion, TestPhraseSuggestions, SynonymTokenizerTest, EchoParamsTest, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestDefaultSimilarityFactory, TestFastWriter, ResourceLoaderTest, URLClassifyProcessorTest, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestTolerantSearch, TestJettySolrRunner, AliasIntegrationTest, CollectionTooManyReplicasTest, HttpPartitionTest, OverseerStatusTest, OverseerTaskQueueTest, RemoteQueryErrorTest, SimpleCollectionCreateDeleteTest, TestCloudInspectUtil, TestCloudPivotFacet, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestLeaderElectionZkExpiry, TestReplicaProperties, TriLevelCompositeIdRoutingTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [437/542] on J1 in 235.81s, 1 test, 1 error <<< FAILURES!

[...truncated 351 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 2 errors, 30 ignored (17 assumptions) [seed: 5CB427BB0F96593A]

Total time: 185 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 987 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/987/

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:38468/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:38468/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([789B750F4A2528FA:F0CF4AD5E4D94502]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10521 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/init-core-data-001
   [junit4]   2> 1166027 INFO  (SUITE-BasicDistributedZkTest-seed#[789B750F4A2528FA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1166032 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1166033 INFO  (Thread-9063) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1166033 INFO  (Thread-9063) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1166133 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.ZkTestServer start zk server on port:57926
   [junit4]   2> 1166133 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1166135 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1166139 INFO  (zkCallback-278-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51b80b1e name:ZooKeeperConnection Watcher:127.0.0.1:57926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1166139 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1166140 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1166140 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1166144 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1166146 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1166160 INFO  (zkCallback-279-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@120b271b name:ZooKeeperConnection Watcher:127.0.0.1:57926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1166161 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1166161 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1166161 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1166163 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1166165 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1166166 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1166168 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1166168 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1166171 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1166171 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1166175 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1166175 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1166179 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1166179 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1166182 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1166183 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1166184 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1166185 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1166193 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1166194 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1166196 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1166197 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1166201 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1166201 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1166204 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1166204 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1166206 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1166206 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1166536 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1
   [junit4]   2> 1166538 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1166540 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@340e3e0{/,null,AVAILABLE}
   [junit4]   2> 1166546 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7231e412{HTTP/1.1}{127.0.0.1:46319}
   [junit4]   2> 1166546 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.Server Started @1169599ms
   [junit4]   2> 1166546 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=46319, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores, hostContext=/}
   [junit4]   2> 1166547 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1166547 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/'
   [junit4]   2> 1166547 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1166547 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1166567 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1166567 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1166570 INFO  (zkCallback-280-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51b6c905 name:ZooKeeperConnection Watcher:127.0.0.1:57926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1166570 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1166570 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1166572 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1166576 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/solr.xml
   [junit4]   2> 1166594 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores
   [junit4]   2> 1166594 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoreContainer New CoreContainer 1715728970
   [junit4]   2> 1166594 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/]
   [junit4]   2> 1166594 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/lib
   [junit4]   2> 1166594 WARN  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/lib).
   [junit4]   2> 1166622 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1166623 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1166623 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1166624 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1166624 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57926/solr
   [junit4]   2> 1166624 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1166624 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1166626 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1166627 INFO  (zkCallback-282-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@737ba97c name:ZooKeeperConnection Watcher:127.0.0.1:57926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1166627 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1166627 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1166636 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1166636 INFO  (zkCallback-283-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3176c77c name:ZooKeeperConnection Watcher:127.0.0.1:57926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1166637 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1166638 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1166641 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1166644 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1166660 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1166665 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1166668 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1166670 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1166673 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1166675 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1166676 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46319_
   [junit4]   2> 1166677 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46319_
   [junit4]   2> 1166679 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1166680 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1166682 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1166682 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94696361185247236-127.0.0.1:46319_-n_0000000000
   [junit4]   2> 1166683 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46319_
   [junit4]   2> 1166683 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1166684 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer Overseer (id=94696361185247236-127.0.0.1:46319_-n_0000000000) starting
   [junit4]   2> 1166689 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1166718 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1166718 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1166721 INFO  (OverseerCollectionConfigSetProcessor-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1166724 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1166746 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1166746 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1166747 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores
   [junit4]   2> 1166748 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/, schema=schema.xml, collection=control_collection, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1, shard=, transient=false, coreNodeName=, config=solrconfig.xml, configSetProperties=configsetprops.json, dataDir=data/}
   [junit4]   2> 1166748 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/
   [junit4]   2> 1166748 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1166750 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1166750 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1166752 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1166752 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:46319_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1166753 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:46319_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:46319",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 1166754 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:46319_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:46319",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null}
   [junit4]   2> 1166755 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1166756 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1166756 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1166759 INFO  (zkCallback-283-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1167757 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1167757 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1167758 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1167758 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1167759 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1167759 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/'
   [junit4]   2> 1167759 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1167759 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1167773 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1167779 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1167793 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1167835 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1167836 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1167843 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1168106 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1168116 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1168119 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1168129 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1168133 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1168136 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1168136 WARN  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1168136 WARN  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1168137 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1168137 WARN  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1168137 WARN  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1168138 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1168138 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1168138 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1168139 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1168139 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1eb42c1
   [junit4]   2> 1168139 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/data
   [junit4]   2> 1168139 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1168140 WARN  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1168140 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/data/index
   [junit4]   2> 1168140 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=1.908203125, floorSegmentMB=0.57421875, forceMergeDeletesPctAllowed=0.6100559745176637, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1168141 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@648a9389 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@45dc3af0),segFN=segments_1,generation=1}
   [junit4]   2> 1168141 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1168141 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/data/
   [junit4]   2> 1168141 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1168145 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1168145 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1168145 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1168145 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1168145 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1168146 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1168146 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1168146 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1168146 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1168149 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1168150 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1168151 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1168152 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1168156 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/file,/admin/luke,/schema,standard,/admin/mbeans,/admin/system,/admin/properties,/update/json,/update/csv,/update,/replication,/admin/ping,/admin/logging,/admin/plugins,/config,/get,/update/json/docs,/admin/threads,/admin/segments
   [junit4]   2> 1168157 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1168158 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1168158 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1168158 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1168158 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1168159 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1168159 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@648a9389 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@45dc3af0),segFN=segments_1,generation=1}
   [junit4]   2> 1168159 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1168159 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@184bb907[collection1] main
   [junit4]   2> 1168159 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1168160 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1168160 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1168161 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1168161 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1168161 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1168161 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1168161 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1168161 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1168162 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1168162 INFO  (searcherExecutor-2136-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@184bb907[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1168163 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1168163 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1168163 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1168163 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515141781196374016
   [junit4]   2> 1168165 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515141781196374016
   [junit4]   2> 1168165 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1168166 INFO  (coreLoadExecutor-2135-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1168166 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46319 collection:control_collection shard:shard1
   [junit4]   2> 1168167 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1168171 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94696361185247236-core_node1-n_0000000000
   [junit4]   2> 1168171 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1168173 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1168173 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1168173 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46319/collection1/
   [junit4]   2> 1168173 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1168173 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46319/collection1/ has no replicas
   [junit4]   2> 1168173 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46319/collection1/ shard1
   [junit4]   2> 1168173 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1168174 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1168177 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1168180 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46319",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1168254 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1168261 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1168262 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4cbde5b7 name:ZooKeeperConnection Watcher:127.0.0.1:57926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1168262 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1168262 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1168262 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1168265 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1168283 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1168283 INFO  (zkCallback-283-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1168331 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46319/collection1/ and leader is http://127.0.0.1:46319/collection1/
   [junit4]   2> 1168331 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46319
   [junit4]   2> 1168331 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1168331 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1168332 INFO  (coreZkRegister-2129-thread-1-processing-c:control_collection n:127.0.0.1:46319_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:46319_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1168334 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:46319_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:46319",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1"} current state version: 2
   [junit4]   2> 1168335 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:46319_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:46319",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 1168438 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1168438 INFO  (zkCallback-283-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1168587 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1
   [junit4]   2> 1168588 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001
   [junit4]   2> 1168588 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1168590 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f6aaaa3{/,null,AVAILABLE}
   [junit4]   2> 1168591 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3aad0a50{HTTP/1.1}{127.0.0.1:38468}
   [junit4]   2> 1168591 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.Server Started @1171644ms
   [junit4]   2> 1168591 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/tempDir-001/jetty1, hostPort=38468}
   [junit4]   2> 1168591 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1168591 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/'
   [junit4]   2> 1168591 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1168591 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1168606 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1168607 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1168608 INFO  (zkCallback-286-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@266ae6e2 name:ZooKeeperConnection Watcher:127.0.0.1:57926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1168609 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1168609 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1168613 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1168613 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/solr.xml
   [junit4]   2> 1168627 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores
   [junit4]   2> 1168627 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoreContainer New CoreContainer 448306696
   [junit4]   2> 1168627 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/]
   [junit4]   2> 1168627 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/lib
   [junit4]   2> 1168627 WARN  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/lib).
   [junit4]   2> 1168635 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1168636 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1168636 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1168637 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1168637 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57926/solr
   [junit4]   2> 1168637 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1168637 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1168638 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1168639 INFO  (zkCallback-288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e3f18bf name:ZooKeeperConnection Watcher:127.0.0.1:57926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1168639 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1168639 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1168641 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1168642 INFO  (zkCallback-289-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e42f903 name:ZooKeeperConnection Watcher:127.0.0.1:57926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1168642 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1168648 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1169654 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38468_
   [junit4]   2> 1169655 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38468_
   [junit4]   2> 1169658 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1169658 INFO  (zkCallback-283-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1169658 INFO  (zkCallback-289-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1169660 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1169660 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94696361185247240-127.0.0.1:38468_-n_0000000001
   [junit4]   2> 1169661 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94696361185247236-127.0.0.1:46319_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1169675 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1169675 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1169675 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores
   [junit4]   2> 1169676 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, coreNodeName=, schema=schema.xml, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/, config=solrconfig.xml, shard=, loadOnStartup=true, transient=false, collection=collection1, configSetProperties=configsetprops.json}
   [junit4]   2> 1169676 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/
   [junit4]   2> 1169676 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1169677 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1169678 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1169678 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1169678 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:38468_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1169680 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:38468_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:38468",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null} current state version: 3
   [junit4]   2> 1169681 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:38468_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:38468",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null}
   [junit4]   2> 1169681 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1169681 INFO  (OverseerStateUpdate-94696361185247236-127.0.0.1:46319_-n_0000000000) [n:127.0.0.1:46319_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1169681 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1169784 INFO  (zkCallback-283-thread-1-processing-n:127.0.0.1:46319_) [n:127.0.0.1:46319_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1169784 INFO  (zkCallback-289-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1169784 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1170682 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1170682 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1170683 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1170683 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1170684 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1170684 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1170684 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1170684 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1170699 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1170705 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1170719 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1170749 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1170750 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1170758 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1171006 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1171014 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1171016 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1171027 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1171030 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1171032 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1171033 WARN  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1171033 WARN  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1171033 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1171034 WARN  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1171034 WARN  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1171035 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1171035 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1171035 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1171035 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1171035 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1eb42c1
   [junit4]   2> 1171036 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1171036 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1171036 WARN  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1171037 INFO  (coreLoadExecutor-2146-thread-1-processing-n:127.0.0.1:38468_) [n:127.0.0.1:38468_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1171037 INFO  (coreLoadE

[...truncated too long message...]

p/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-4-001/cores/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-4-001/cores/collection1/data/index;done=false>>]
   [junit4]   2> 1195681 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:36087_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001/shard-4-001/cores/collection1/data/index
   [junit4]   2> 1195705 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:36087_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.Overseer Overseer (id=94696361185247249-127.0.0.1:36087_-n_0000000004) closing
   [junit4]   2> 1195705 INFO  (OverseerStateUpdate-94696361185247249-127.0.0.1:36087_-n_0000000004) [n:127.0.0.1:36087_    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36087_
   [junit4]   2> 1195714 INFO  (zkCallback-302-thread-2-processing-n:127.0.0.1:36087_) [n:127.0.0.1:36087_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1195715 WARN  (zkCallback-302-thread-2-processing-n:127.0.0.1:36087_) [n:127.0.0.1:36087_    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1195732 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@604e5e8f{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 1195733 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@296ddb2c{/,null,UNAVAILABLE}
   [junit4]   2> 1195735 INFO  (TEST-BasicDistributedZkTest.test-seed#[789B750F4A2528FA]) [n:127.0.0.1:36087_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:57926 57926
   [junit4]   2> 1195907 INFO  (Thread-9063) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:57926 57926
   [junit4]   2> 1195908 WARN  (Thread-9063) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/aliases.json
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/collections
   [junit4]   2> 	7	/solr/live_nodes
   [junit4]   2> 	5	/solr/overseer/queue-work
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 	5	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.method=test -Dtests.seed=789B750F4A2528FA -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=da -Dtests.timezone=America/Godthab -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 29.9s J0 | BasicDistributedZkTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:38468/collection1 expected:<68> but was:<67>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([789B750F4A2528FA:F0CF4AD5E4D94502]:0)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1195925 INFO  (SUITE-BasicDistributedZkTest-seed#[789B750F4A2528FA]-worker) [n:127.0.0.1:36087_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_789B750F4A2528FA-001
   [junit4]   2> Oct 15, 2015 7:30:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {n_tf1=PostingsFormat(name=LuceneVarGapDocFreqInterval), oddField_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_i1=PostingsFormat(name=LuceneVarGapDocFreqInterval), foo_b=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval), n_dt1=Lucene50(blocksize=128), intDefault=Lucene50(blocksize=128), range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_f1=Lucene50(blocksize=128), rnd_b=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_ti1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), foo_i=Lucene50(blocksize=128), _version_=Lucene50(blocksize=128), n_tl1=Lucene50(blocksize=128), foo_f=PostingsFormat(name=Memory doPackFST= false), n_td1=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval), SubjectTerms_mfacet=Lucene50(blocksize=128), foo_d=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_tdt1=Lucene50(blocksize=128), a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), n_d1=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), other_tl1=Lucene50(blocksize=128), n_l1=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=da, timezone=America/Godthab
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=213125480,total=425197568
   [junit4]   2> NOTE: All tests run in this JVM: [DocumentAnalysisRequestHandlerTest, ReplicationFactorTest, SegmentsInfoRequestHandlerTest, JavabinLoaderTest, AlternateDirectoryTest, ExternalFileFieldSortTest, TestDocSet, TestReplicationHandlerBackup, TestDistributedMissingSort, RequestLoggingTest, SuggestComponentTest, TestBinaryField, JsonLoaderTest, IndexSchemaRuntimeFieldTest, TestFieldResource, PingRequestHandlerTest, TestCSVLoader, CurrencyFieldOpenExchangeTest, TestComponentsName, TestArbitraryIndexDir, StandardRequestHandlerTest, TestAnalyzeInfixSuggestions, TestDocumentBuilder, PeerSyncTest, SpatialRPTFieldTypeTest, EnumFieldTest, TestManagedSchema, TestIBSimilarityFactory, AddSchemaFieldsUpdateProcessorFactoryTest, OpenExchangeRatesOrgProviderTest, SpellingQueryConverterTest, TestSerializedLuceneMatchVersion, TestReplicationHandler, DateMathParserTest, AsyncCallRequestStatusResponseTest, TestJoin, TestDownShardTolerantSearch, TestDynamicFieldResource, DisMaxRequestHandlerTest, TestRuleBasedAuthorizationPlugin, BadCopyFieldTest, TestFoldingMultitermQuery, PolyFieldTest, ResponseLogComponentTest, IndexSchemaTest, TestDistributedStatsComponentCardinality, TestCollapseQParserPlugin, DefaultValueUpdateProcessorTest, HdfsSyncSliceTest, TestConfig, BasicDistributedZkTest]
   [junit4] Completed [172/542] on J0 in 30.95s, 1 test, 1 failure <<< FAILURES!

[...truncated 1203 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2142 tests, 1 failure, 30 ignored (17 assumptions) [seed: 789B750F4A2528FA]

Total time: 194 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 986 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/986/

6 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=19876, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19876, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:52277/ib_un/i, http://127.0.0.1:34349/ib_un/i, http://127.0.0.1:49462/ib_un/i, http://127.0.0.1:59204/ib_un/i, http://127.0.0.1:56448/ib_un/i]
	at __randomizedtesting.SeedInfo.seed([56CDA7C6361D5707]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:52277/ib_un/i, http://127.0.0.1:34349/ib_un/i, http://127.0.0.1:49462/ib_un/i, http://127.0.0.1:59204/ib_un/i, http://127.0.0.1:56448/ib_un/i]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:52277/ib_un/i: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
89 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=24698, name=coreZkRegister-4119-thread-16, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=19686, name=searcherExecutor-4142-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=19500, name=SessionTracker, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)    4) Thread[id=24701, name=coreZkRegister-4119-thread-19, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=23722, name=qtp702287161-23722, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=19746, name=searcherExecutor-4196-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=22777, name=qtp2121314754-22777, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=21996, name=qtp2121314754-21996, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=23525, name=coreZkRegister-4108-thread-4-processing-r:core_node1 s:shard1 n:127.0.0.1:59204_ib_un%2Fi x:awholynewcollection_6_shard1_replica1 c:awholynewcollection_6, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=19844, name=searcherExecutor-4258-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=23506, name=coreZkRegister-4119-thread-14-processing-r:core_node18 s:shard9 n:127.0.0.1:56448_ib_un%2Fi x:awholynewstresscollection_collection5_7_shard9_replica2 c:awholynewstresscollection_collection5_7, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1071)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1024)         at org.apache.solr.cloud.ZkController.register(ZkController.java:938)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=24837, name=zkCallback-2277-thread-33, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=23228, name=qtp702287161-23228, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=19682, name=searcherExecutor-4132-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=19638, name=qtp702287161-19638-selector-ServerConnectorManager@2fc4d021/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=23724, name=qtp702287161-23724, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=19870, name=searcherExecutor-4275-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=23534, name=coreZkRegister-4108-thread-9-processing-r:core_node1 s:shard1 n:127.0.0.1:59204_ib_un%2Fi x:awholynewcollection_6_shard1_replica1 c:awholynewcollection_6, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=19615, name=org.eclipse.jetty.server.session.HashSessionManager@4a606520Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=24845, name=zkCallback-2277-thread-36, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=19499, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=24840, name=zkCallback-2277-thread-34, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=19639, name=qtp702287161-19639-acceptor-0@214df620-ServerConnector@223c3f07{HTTP/1.1}{127.0.0.1:56448}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=23143, name=qtp702287161-23143, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=23203, name=qtp702287161-23203, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=24699, name=coreZkRegister-4119-thread-17, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=21624, name=qtp702287161-21624, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=24668, name=zkCallback-2272-thread-10-processing-n:127.0.0.1:59204_ib_un%2Fi-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   29) Thread[id=19726, name=searcherExecutor-4176-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=24864, name=zkCallback-2272-thread-44, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=23151, name=qtp2121314754-23151, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=19661, name=searcherExecutor-4125-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=24859, name=coreZkRegister-4108-thread-18-processing-r:core_node1 s:shard1 n:127.0.0.1:59204_ib_un%2Fi x:awholynewcollection_6_shard1_replica1 c:awholynewcollection_6, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=23505, name=coreZkRegister-4119-thread-13-processing-r:core_node18 s:shard9 n:127.0.0.1:56448_ib_un%2Fi x:awholynewstresscollection_collection5_7_shard9_replica2 c:awholynewstresscollection_collection5_7, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1071)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1024)         at org.apache.solr.cloud.ZkController.register(ZkController.java:938)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=19753, name=searcherExecutor-4206-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=19865, name=searcherExecutor-4267-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=19608, name=qtp2121314754-19608-acceptor-0@19b169c8-ServerConnector@17504ff5{HTTP/1.1}{127.0.0.1:59204}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=20040, name=qtp2121314754-20040, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=24856, name=coreZkRegister-4108-thread-15-processing-r:core_node1 s:shard1 n:127.0.0.1:59204_ib_un%2Fi x:awholynewcollection_6_shard1_replica1 c:awholynewcollection_6, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=23504, name=coreZkRegister-4119-thread-12-processing-r:core_node18 s:shard9 n:127.0.0.1:56448_ib_un%2Fi x:awholynewstresscollection_collection5_7_shard9_replica2 c:awholynewstresscollection_collection5_7, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1071)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1024)         at org.apache.solr.cloud.ZkController.register(ZkController.java:938)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=19637, name=qtp702287161-19637-selector-ServerConnectorManager@2fc4d021/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=23711, name=qtp2121314754-23711, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=19626, name=Thread-10799, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1155)   44) Thread[id=19612, name=qtp2121314754-19612, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=24841, name=zkCallback-2277-thread-35, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=21997, name=qtp2121314754-21997, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=19614, name=Scheduler-1655446912, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=20724, name=qtp2121314754-20724, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=24836, name=zkCallback-2277-thread-32, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=20228, name=qtp2121314754-20228, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=19867, name=searcherExecutor-4264-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=23725, name=qtp702287161-23725, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=23150, name=qtp2121314754-23150, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=24848, name=OverseerCollectionConfigSetProcessor-94691446703194251-127.0.0.1:56448_ib_un%2Fi-n_0000000020, state=TIMED_WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:344)         at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:223)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:197)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=19841, name=searcherExecutor-4253-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=24276, name=zkCallback-2277-thread-10-processing-n:127.0.0.1:56448_ib_un%2Fi-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   57) Thread[id=24847, name=OverseerStateUpdate-94691446703194251-127.0.0.1:56448_ib_un%2Fi-n_0000000020, state=TIMED_WAITING, group=Overseer state updater.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:170)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:144)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:193)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=19606, name=qtp2121314754-19606-selector-ServerConnectorManager@768cd281/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=24846, name=zkCallback-2277-thread-37, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=19657, name=Thread-10813, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1155)   61) Thread[id=24860, name=coreZkRegister-4108-thread-19, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=22316, name=qtp2121314754-22316, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=19672, name=Scheduler-413562984, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   64) Thread[id=24854, name=zkCallback-2272-thread-43, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   65) Thread[id=19502, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)   66) Thread[id=24706, name=coreZkRegister-4119-thread-23, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=22035, name=qtp2121314754-22035, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   68) Thread[id=24857, name=coreZkRegister-4108-thread-16, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=19498, name=Thread-10750, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Thread.join(Thread.java:1281)         at java.lang.Thread.join(Thread.java:1355)         at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)   70) Thread[id=19607, name=qtp2121314754-19607-selector-ServerConnectorManager@768cd281/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=24861, name=coreZkRegister-4108-thread-20, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   72) Thread[id=22307, name=qtp2121314754-22307, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   73) Thread[id=19674, name=Scheduler-770191015, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   74) Thread[id=23524, name=coreZkRegister-4108-thread-3-processing-r:core_node1 s:shard1 n:127.0.0.1:59204_ib_un%2Fi x:awholynewcollection_6_shard1_replica1 c:awholynewcollection_6, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   75) Thread[id=24275, name=zkCallback-2277-thread-10-processing-n:127.0.0.1:56448_ib_un%2Fi-SendThread(127.0.0.1:41313), state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   76) Thread[id=24849, name=OverseerHdfsCoreFailoverThread-94691446703194251-127.0.0.1:56448_ib_un%2Fi-n_0000000020, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)         at java.lang.Thread.run(Thread.java:745)   77) Thread[id=19646, name=org.eclipse.jetty.server.session.HashSessionManager@4676f36aTimer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   78) Thread[id=19630, name=searcherExecutor-4114-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   79) Thread[id=19728, name=searcherExecutor-4175-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   80) Thread[id=23142, name=qtp702287161-23142, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   81) Thread[id=24667, name=zkCallback-2272-thread-10-processing-n:127.0.0.1:59204_ib_un%2Fi-SendThread(127.0.0.1:41313), state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   82) Thread[id=19645, name=Scheduler-1740826641, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   83) Thread[id=19862, name=searcherExecutor-4263-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   84) Thread[id=24855, name=coreZkRegister-4108-thread-14-processing-r:core_node1 s:shard1 n:127.0.0.1:59204_ib_un%2Fi x:awholynewcollection_6_shard1_replica1 c:awholynewcollection_6, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   85) Thread[id=24710, name=coreZkRegister-4119-thread-25, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   86) Thread[id=22014, name=qtp702287161-22014, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   87) Thread[id=23503, name=coreZkRegister-4119-thread-11-processing-r:core_node18 s:shard9 n:127.0.0.1:56448_ib_un%2Fi x:awholynewstresscollection_collection5_7_shard9_replica2 c:awholynewstresscollection_collection5_7, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1071)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1024)         at org.apache.solr.cloud.ZkController.register(ZkController.java:938)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   88) Thread[id=24709, name=coreZkRegister-4119-thread-24, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   89) Thread[id=24858, name=coreZkRegister-4108-thread-17, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 89 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=24698, name=coreZkRegister-4119-thread-16, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=19686, name=searcherExecutor-4142-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=19500, name=SessionTracker, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   4) Thread[id=24701, name=coreZkRegister-4119-thread-19, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=23722, name=qtp702287161-23722, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=19746, name=searcherExecutor-4196-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=22777, name=qtp2121314754-22777, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=21996, name=qtp2121314754-21996, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=23525, name=coreZkRegister-4108-thread-4-processing-r:core_node1 s:shard1 n:127.0.0.1:59204_ib_un%2Fi x:awholynewcollection_6_shard1_replica1 c:awholynewcollection_6, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)
        at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)
        at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
        at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
        at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
        at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
        at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=19844, name=searcherExecutor-4258-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=23506, name=coreZkRegister-4119-thread-14-processing-r:core_node18 s:shard9 n:127.0.0.1:56448_ib_un%2Fi x:awholynewstresscollection_collection5_7_shard9_replica2 c:awholynewstresscollection_collection5_7, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1071)
        at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)
        at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1024)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:938)
        at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=24837, name=zkCallback-2277-thread-33, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=23228, name=qtp702287161-23228, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=19682, name=searcherExecutor-4132-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=19638, name=qtp702287161-19638-selector-ServerConnectorManager@2fc4d021/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.nio.ch.EPollArrayWrapper.epollWa

[...truncated too long message...]

java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25640, name=coreZkRegister-7731-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([56CDA7C6361D5707]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4007489 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[56CDA7C6361D5707]-worker) [n:127.0.0.1:37553_vj_jol c:all_in_lir s:shard1 r:core_node1 x:all_in_lir_shard1_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_56CDA7C6361D5707-001
   [junit4]   2> Oct 14, 2015 11:07:58 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ms_MY, timezone=Asia/Karachi
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=245458264,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [UpdateRequestProcessorFactoryTest, TestShortCircuitedRequests, ConvertedLegacyTest, TestQueryTypes, FastVectorHighlighterTest, TestSchemaResource, SimpleMLTQParserTest, TestMiniSolrCloudCluster, LoggingHandlerTest, TestChildDocTransformer, TestSolrCoreProperties, TestTrieFacet, MoreLikeThisHandlerTest, DistribDocExpirationUpdateProcessorTest, TestAuthenticationFramework, TestTrackingShardHandlerFactory, MBeansHandlerTest, TestScoreJoinQPNoScore, CursorPagingTest, DistributedFacetPivotSmallTest, TestObjectReleaseTracker, SuggesterFSTTest, ZkControllerTest, TestConfigSetsAPI, InfoHandlerTest, SolrXmlInZkTest, DateRangeFieldTest, PKIAuthenticationIntegrationTest, TestNRTOpen, TestManagedSchemaDynamicFieldResource, SpellPossibilityIteratorTest, PreAnalyzedFieldTest, TestBulkSchemaConcurrent, DirectSolrSpellCheckerTest, URLClassifyProcessorTest, TestSha256AuthenticationProvider, HdfsWriteToMultipleCollectionsTest, ZkCLITest, TestReloadDeadlock, WordBreakSolrSpellCheckerTest, SharedFSAutoReplicaFailoverTest, TestPartialUpdateDeduplication, AliasIntegrationTest, ShardRoutingCustomTest, TestDynamicLoading, TestDistributedGrouping, SpatialFilterTest, MinimalSchemaTest, TestBinaryResponseWriter, HLLSerializationTest, TestFileDictionaryLookup, SimplePostToolTest, CollectionTooManyReplicasTest, TestCryptoKeys, ExplicitHLLTest, TestSuggestSpellingConverter, BlockCacheTest, TermVectorComponentTest, TestSolrXml, MergeStrategyTest, TestUtils, DOMUtilTest, TestRestoreCore, TestRebalanceLeaders, DocValuesTest, DistributedIntervalFacetingTest, SoftAutoCommitTest, ShardRoutingTest, SolrInfoMBeanTest, TestLMJelinekMercerSimilarityFactory, RuleEngineTest, TestRandomRequestDistribution, TestUpdate, SolrIndexSplitterTest, SaslZkACLProviderTest, ZkSolrClientTest, OpenCloseCoreStressTest, IndexBasedSpellCheckerTest, TestFastOutputStream, CustomCollectionTest, RecoveryAfterSoftCommitTest, MultiThreadedOCPTest, CircularListTest, TestFiltering, TestDFRSimilarityFactory, TestSurroundQueryParser, TestRTGBase, ConcurrentDeleteAndCreateCollectionTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, ClusterStateUpdateTest, TestRandomFaceting, TestRandomDVFaceting, AutoCommitTest, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestReload, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, SpellCheckComponentTest, TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, BadIndexSchemaTest, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, HighlighterTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SpellCheckCollatorTest, CoreAdminHandlerTest, TestFoldingMultitermQuery, TestStressLucene, TestTrie, TestCSVLoader, PolyFieldTest, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, TestRemoteStreaming, DistanceFunctionTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, LukeRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, TermsComponentTest, RequestHandlersTest, DocumentBuilderTest, TestIndexingPerformance, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, QueryParsingTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, SearchHandlerTest, TestLFUCache, SOLR749Test, TestQuerySenderListener, UpdateParamsTest, TestSolrIndexConfig, TestQuerySenderNoQuery, ResponseLogComponentTest, SolrIndexConfigTest, TestStressRecovery, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, TestElisionMultitermQuery, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCodecSupport, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, ResourceLoaderTest, PluginInfoTest, DateMathParserTest, TestSystemIdResolver, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, DistributedMLTComponentTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, ActionThrottleTest, AssignTest, AsyncCallRequestStatusResponseTest, CollectionStateFormat2Test, CollectionsAPIAsyncDistributedZkTest, DeleteInactiveReplicaTest, DistributedVersionInfoTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [488/542] on J1 in 368.13s, 1 test, 1 error <<< FAILURES!

[...truncated 195 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2142 tests, 2 suite-level errors, 3 errors, 1 failure, 395 ignored (17 assumptions) [seed: 56CDA7C6361D5707]

Total time: 206 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 985 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/985/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=23846, name=coreZkRegister-5380-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=23846, name=coreZkRegister-5380-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([FF3E80D22C005140]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=28964, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=28964, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45446/z_: Could not find collection : awholynewstresscollection_collection5_2
	at __randomizedtesting.SeedInfo.seed([FF3E80D22C005140]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11209 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/init-core-data-001
   [junit4]   2> 2659898 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[FF3E80D22C005140]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2659902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2659902 INFO  (Thread-15055) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2659903 INFO  (Thread-15055) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2660003 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.ZkTestServer start zk server on port:58055
   [junit4]   2> 2660003 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2660003 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2660006 INFO  (zkCallback-2125-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ed96ed1 name:ZooKeeperConnection Watcher:127.0.0.1:58055 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2660006 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2660006 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2660007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2660009 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2660010 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2660011 INFO  (zkCallback-2126-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27b7842a name:ZooKeeperConnection Watcher:127.0.0.1:58055/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2660011 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2660011 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2660011 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2660013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2660015 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2660016 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2660017 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2660017 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2660020 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2660020 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2660022 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2660022 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2660023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2660023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2660025 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2660025 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2660026 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2660027 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2660028 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2660028 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2660030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2660030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2660031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2660032 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2660033 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2660033 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2660035 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2660035 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2660334 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1
   [junit4]   2> 2660336 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2660337 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19467583{/,null,AVAILABLE}
   [junit4]   2> 2660339 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.ServerConnector Started ServerConnector@30c96a73{HTTP/1.1}{127.0.0.1:37465}
   [junit4]   2> 2660340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.Server Started @2663179ms
   [junit4]   2> 2660340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores, hostPort=37465}
   [junit4]   2> 2660340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 2660340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/'
   [junit4]   2> 2660341 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2660341 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2660356 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2660356 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2660357 INFO  (zkCallback-2127-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12e4518f name:ZooKeeperConnection Watcher:127.0.0.1:58055/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2660358 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2660358 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2660359 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2660359 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/solr.xml
   [junit4]   2> 2660372 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores
   [junit4]   2> 2660372 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoreContainer New CoreContainer 838903967
   [junit4]   2> 2660372 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/]
   [junit4]   2> 2660372 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/lib
   [junit4]   2> 2660373 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/lib).
   [junit4]   2> 2660380 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2660381 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2660382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2660382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2660382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58055/solr
   [junit4]   2> 2660382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2660382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2660383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2660384 INFO  (zkCallback-2129-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c0ebb7c name:ZooKeeperConnection Watcher:127.0.0.1:58055 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2660385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2660385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2660386 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2660387 INFO  (zkCallback-2130-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@93a8571 name:ZooKeeperConnection Watcher:127.0.0.1:58055/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2660388 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2660388 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2660390 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2660393 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2660395 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2660396 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2660398 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2660400 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2660401 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2660402 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2660403 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37465_
   [junit4]   2> 2660403 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37465_
   [junit4]   2> 2660404 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2660406 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2660407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2660407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94685139554664452-127.0.0.1:37465_-n_0000000000
   [junit4]   2> 2660408 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37465_
   [junit4]   2> 2660408 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2660409 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer Overseer (id=94685139554664452-127.0.0.1:37465_-n_0000000000) starting
   [junit4]   2> 2660410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2660415 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2660416 INFO  (OverseerCollectionConfigSetProcessor-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2660416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2660416 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2660428 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2660428 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2660429 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores
   [junit4]   2> 2660430 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, transient=false, name=collection1, schema=schema.xml, configSetProperties=configsetprops.json, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1, dataDir=data/, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/, collection=control_collection, loadOnStartup=true}
   [junit4]   2> 2660430 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/
   [junit4]   2> 2660430 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2660431 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2660431 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:37465_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2660431 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2660431 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2660433 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37465",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:37465_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 2660433 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37465",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:37465_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 2660433 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2660434 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2660434 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2660435 INFO  (zkCallback-2130-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2661434 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2661434 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2661435 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2661435 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2661435 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2661435 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/'
   [junit4]   2> 2661436 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2661436 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2661453 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2661460 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2661475 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2661505 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2661506 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2661513 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2661787 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2661795 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2661797 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2661806 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2661809 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2661812 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2661813 WARN  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2661813 WARN  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2661813 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2661814 WARN  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2661814 WARN  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2661815 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2661815 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2661815 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2661816 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2661816 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@55d46b4
   [junit4]   2> 2661817 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/data
   [junit4]   2> 2661817 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2661817 WARN  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2661817 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/data/index
   [junit4]   2> 2661818 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=257468062, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2661818 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@1a7e5344 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5b8ca2f2; maxCacheMB=0.59375 maxMergeSizeMB=0.919921875)),segFN=segments_1,generation=1}
   [junit4]   2> 2661818 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2661819 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/data/
   [junit4]   2> 2661819 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2661823 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2661824 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2661824 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2661824 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2661824 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2661824 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2661824 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2661825 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2661825 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2661828 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2661829 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2661831 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2661832 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2661835 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,/admin/ping,/admin/system,/admin/luke,/admin/logging,/admin/segments,/admin/threads,/admin/properties,standard,/replication,/admin/plugins,/admin/mbeans,/get,/admin/file,/update,/schema,/config,/update/csv,/update/json
   [junit4]   2> 2661837 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2661837 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2661838 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2661838 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2661838 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2661839 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=33.0634765625, floorSegmentMB=2.181640625, forceMergeDeletesPctAllowed=26.20650633088077, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.16393690538443745
   [junit4]   2> 2661839 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@1a7e5344 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5b8ca2f2; maxCacheMB=0.59375 maxMergeSizeMB=0.919921875)),segFN=segments_1,generation=1}
   [junit4]   2> 2661839 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2661840 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1e31ddfd[collection1] main
   [junit4]   2> 2661840 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2661841 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2661841 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2661841 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2661841 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2661841 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2661842 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2661842 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2661842 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2661842 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2661843 INFO  (searcherExecutor-5332-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1e31ddfd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2661843 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2661843 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2661843 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2661843 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514962234907820032
   [junit4]   2> 2661845 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514962234907820032
   [junit4]   2> 2661845 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2661846 INFO  (coreLoadExecutor-5331-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2661846 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:37465 collection:control_collection shard:shard1
   [junit4]   2> 2661847 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2661851 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94685139554664452-core_node1-n_0000000000
   [junit4]   2> 2661851 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2661853 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2661853 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2661853 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37465/collection1/
   [junit4]   2> 2661853 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2661853 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37465/collection1/ has no replicas
   [junit4]   2> 2661853 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37465/collection1/ shard1
   [junit4]   2> 2661853 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2661853 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2661856 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2661858 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37465",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2661933 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2661934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2661935 INFO  (zkCallback-2132-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1075f98a name:ZooKeeperConnection Watcher:127.0.0.1:58055/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2661936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2661936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2661936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2661939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2661961 INFO  (zkCallback-2130-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2661961 INFO  (zkCallback-2132-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2662009 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:37465/collection1/ and leader is http://127.0.0.1:37465/collection1/
   [junit4]   2> 2662010 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37465
   [junit4]   2> 2662010 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2662010 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2662010 INFO  (coreZkRegister-5325-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:37465_ x:collection1) [n:127.0.0.1:37465_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2662013 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37465",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:37465_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1"} current state version: 2
   [junit4]   2> 2662013 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37465",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:37465_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 2662116 INFO  (zkCallback-2130-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2662116 INFO  (zkCallback-2132-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2662262 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores/collection1
   [junit4]   2> 2662263 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001
   [junit4]   2> 2662263 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2662265 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@78fce398{/,null,AVAILABLE}
   [junit4]   2> 2662266 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.ServerConnector Started ServerConnector@29ccd0e{HTTP/1.1}{127.0.0.1:51154}
   [junit4]   2> 2662266 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.e.j.s.Server Started @2665105ms
   [junit4]   2> 2662266 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/tempDir-001/jetty1, hostPort=51154, solrconfig=solrconfig.xml, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores}
   [junit4]   2> 2662267 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 2662267 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/'
   [junit4]   2> 2662267 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2662267 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2662284 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2662285 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2662286 INFO  (zkCallback-2133-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@61ded1f2 name:ZooKeeperConnection Watcher:127.0.0.1:58055/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2662286 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2662287 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2662288 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15063a1fcaa0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2662288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2662288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/solr.xml
   [junit4]   2> 2662303 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores
   [junit4]   2> 2662303 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoreContainer New CoreContainer 2061206599
   [junit4]   2> 2662303 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/]
   [junit4]   2> 2662303 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/lib
   [junit4]   2> 2662303 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/lib).
   [junit4]   2> 2662312 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2662313 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2662314 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2662314 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2662314 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58055/solr
   [junit4]   2> 2662314 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2662314 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2662315 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2662316 INFO  (zkCallback-2135-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@115dc63e name:ZooKeeperConnection Watcher:127.0.0.1:58055 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2662318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2662318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2662319 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15063a1fcaa0007, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2662320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2662321 INFO  (zkCallback-2136-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3fbca303 name:ZooKeeperConnection Watcher:127.0.0.1:58055/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2662321 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2662326 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2663332 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51154_
   [junit4]   2> 2663333 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51154_
   [junit4]   2> 2663334 INFO  (zkCallback-2130-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2663334 INFO  (zkCallback-2136-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2663334 INFO  (zkCallback-2132-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2663335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2663336 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94685139554664456-127.0.0.1:51154_-n_0000000001
   [junit4]   2> 2663337 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94685139554664452-127.0.0.1:37465_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2663350 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2663350 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2663351 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores
   [junit4]   2> 2663351 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, collection=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores/collection1, config=solrconfig.xml, configSetProperties=configsetprops.json, transient=false, shard=, schema=schema.xml, coreNodeName=, name=collection1, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores/collection1/}
   [junit4]   2> 2663352 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores/collection1/
   [junit4]   2> 2663352 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2663353 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2663353 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FF3E80D22C005140]) [n:127.0.0.1:51154_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2663353 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2663353 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2663355 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2663355 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51154",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51154_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1"} current state version: 3
   [junit4]   2> 2663355 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51154",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51154_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 2663355 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2663355 INFO  (OverseerStateUpdate-94685139554664452-127.0.0.1:37465_-n_0000000000) [n:127.0.0.1:37465_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2663458 INFO  (zkCallback-2132-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2663458 INFO  (zkCallback-2130-thread-1-processing-n:127.0.0.1:37465_) [n:127.0.0.1:37465_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2663458 INFO  (zkCallback-2136-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2664355 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2664355 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2664356 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2664356 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 2664357 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2664357 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FF3E80D22C005140-001/shard-1-001/cores/collection1/'
   [junit4]   2> 2664357 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2664357 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2664374 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2664380 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2664394 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2664424 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2664425 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2664432 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2664728 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2664736 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2664738 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2664748 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2664751 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2664754 INFO  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2664755 WARN  (coreLoadExecutor-5342-thread-1-processing-n:127.0.0.1:51154_) [n:127.0.0.1:51154_ c:collection1   x:collection1] o.a.s

[...truncated too long message...]

rMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3721239 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[FF3E80D22C005140]-worker) [n:127.0.0.1:37258_z_ c:addReplicaColl s:shard2 r:core_node2 x:addReplicaColl_shard2_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Oct 13, 2015 8:13:53 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {timestamp=PostingsFormat(name=Memory doPackFST= true), range_facet_l=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), id=FSTOrd50, _version_=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=Memory doPackFST= true), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=FSTOrd50, multiDefault=PostingsFormat(name=Direct), a_i1=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=Memory doPackFST= true)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_GT, timezone=Indian/Mayotte
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=143683176,total=523239424
   [junit4]   2> NOTE: All tests run in this JVM: [ClusterStateTest, StressHdfsTest, TestHighFrequencyDictionaryFactory, SuggesterTest, OverseerTest, TestSortByMinMaxFunction, CloudExitableDirectoryReaderTest, UpdateParamsTest, LukeRequestHandlerTest, TestLRUCache, CleanupOldIndexTest, DistributedQueryComponentCustomSortTest, TestRequestStatusCollectionAPI, TestStressReorder, FieldAnalysisRequestHandlerTest, TestRecovery, ZkNodePropsTest, TestCSVResponseWriter, SolrCloudExampleTest, TestManagedSynonymFilterFactory, TestCoreDiscovery, TestSmileRequest, TestInitParams, TestSort, TestWriterPerf, RequestHandlersTest, TestCloudSchemaless, AnalyticsMergeStrategyTest, TestSearchPerf, TestExpandComponent, TestSolrQueryParserResource, StatsComponentTest, TestFuzzyAnalyzedSuggestions, DistributedDebugComponentTest, TestRandomMergePolicy, TestRemoveLastDynamicCopyField, OverriddenZkACLAndCredentialsProvidersTest, ShowFileRequestHandlerTest, CoreAdminHandlerTest, UUIDFieldTest, TestManagedResource, OverseerRolesTest, TestPKIAuthenticationPlugin, BasicDistributedZk2Test, ChangedSchemaMergeTest, PluginInfoTest, TestCloudManagedSchemaConcurrent, RAMDirectoryFactoryTest, DocValuesMissingTest, TestSystemIdResolver, ZkStateWriterTest, TestSolrIndexConfig, BadIndexSchemaTest, DataDrivenBlockJoinTest, TestSchemaSimilarityResource, TestMaxScoreQueryParser, BadComponentTest, BinaryUpdateRequestHandlerTest, TestConfigSetsAPIExclusivity, SearchHandlerTest, HighlighterMaxOffsetTest, TestDynamicFieldCollectionResource, TestDocBasedVersionConstraints, TestExtendedDismaxParser, ZkStateReaderTest, TestCustomSort, CollectionReloadTest, TestIndexSearcher, StatelessScriptUpdateProcessorFactoryTest, DeleteLastCustomShardedReplicaTest, TestPivotHelperCode, DistributedQueueTest, TestFastLRUCache, TestReloadAndDeleteDocs, SystemInfoHandlerTest, TestFastWriter, TestMacros, RulesTest, ConnectionManagerTest, SolrCoreCheckLockOnStartupTest, TestExceedMaxTermLength, ExitableDirectoryReaderTest, TestCustomDocTransformer, TestSchemaManager, TestReload, DistribCursorPagingTest, TestCollationField, TestRawResponseWriter, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, CollectionStateFormat2Test, DeleteReplicaTest, DistribJoinFromCollectionTest, LeaderInitiatedRecoveryOnShardRestartTest, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDownShardTolerantSearch, TestMiniSolrCloudClusterBase, TriLevelCompositeIdRoutingTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [468/542] on J2 in 496.29s, 1 test, 1 error <<< FAILURES!

[...truncated 253 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2142 tests, 2 errors, 30 ignored (17 assumptions) [seed: FF3E80D22C005140]

Total time: 195 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 984 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/984/

1 tests failed.
FAILED:  org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels

Error Message:
this writer hit an unrecoverable error; cannot commit

Stack Trace:
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot commit
	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2775)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2961)
	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1080)
	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1123)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
	at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:396)
	at org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels(TestBlockJoin.java:1672)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:80)
	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:53)
	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:142)
	at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
	at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:180)
	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4124)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)




Build Log:
[...truncated 7428 lines...]
   [junit4] Suite: org.apache.lucene.search.join.TestBlockJoin
   [junit4]   2> okt. 13, 2015 6:45:33 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #86,5,TGRP-TestBlockJoin]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([9FEAD3AD3FFCFC8E]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:80)
   [junit4]   2> 	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:53)
   [junit4]   2> 	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]   2> 	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]   2> 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:142)
   [junit4]   2> 	at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]   2> 	at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:180)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4124)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> okt. 13, 2015 6:45:33 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #73,5,TGRP-TestBlockJoin]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([9FEAD3AD3FFCFC8E]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:80)
   [junit4]   2> 	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:53)
   [junit4]   2> 	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]   2> 	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]   2> 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:142)
   [junit4]   2> 	at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:127)
   [junit4]   2> 	at org.apache.lucene.store.RateLimitedIndexOutput.writeByte(RateLimitedIndexOutput.java:66)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:40)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.write(SimpleTextFieldsWriter.java:188)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.write(SimpleTextFieldsWriter.java:137)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.write(SimpleTextFieldsWriter.java:58)
   [junit4]   2> 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestBlockJoin -Dtests.method=testMultiChildQueriesOfDiffParentLevels -Dtests.seed=9FEAD3AD3FFCFC8E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=is -Dtests.timezone=Atlantic/Madeira -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    181s J1 | TestBlockJoin.testMultiChildQueriesOfDiffParentLevels <<<
   [junit4]    > Throwable #1: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot commit
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2775)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2961)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1080)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1123)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:396)
   [junit4]    > 	at org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels(TestBlockJoin.java:1672)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:80)
   [junit4]    > 	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:53)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    > 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:142)
   [junit4]    > 	at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]    > 	at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]    > 	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:180)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4124)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=98, name=Lucene Merge Thread #86, state=RUNNABLE, group=TGRP-TestBlockJoin]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9FEAD3AD3FFCFC8E]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:80)
   [junit4]    > 	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:53)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    > 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:142)
   [junit4]    > 	at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]    > 	at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]    > 	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:180)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4124)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=85, name=Lucene Merge Thread #73, state=RUNNABLE, group=TGRP-TestBlockJoin]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9FEAD3AD3FFCFC8E]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:80)
   [junit4]    > 	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:53)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    > 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:142)
   [junit4]    > 	at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:127)
   [junit4]    > 	at org.apache.lucene.store.RateLimitedIndexOutput.writeByte(RateLimitedIndexOutput.java:66)
   [junit4]    > 	at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
   [junit4]    > 	at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:40)
   [junit4]    > 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.write(SimpleTextFieldsWriter.java:188)
   [junit4]    > 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.write(SimpleTextFieldsWriter.java:137)
   [junit4]    > 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.write(SimpleTextFieldsWriter.java:58)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/test/J1/temp/lucene.search.join.TestBlockJoin_9FEAD3AD3FFCFC8E-001
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {skill=DFR I(F)LZ(0.3), child=DFR GL1, parent=DFR I(n)L2, country=DFR I(F)BZ(0.3), docType=DFR I(ne)L1, qualification=IB SPL-D3(800.0), childText=DFR GB1, parentText=DFR I(ne)B2}, locale=is, timezone=Atlantic/Madeira
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=67196488,total=441450496
   [junit4]   2> NOTE: All tests run in this JVM: [TestBlockJoin]
   [junit4] Completed [6/6] on J1 in 194.66s, 19 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/test/temp/junit4-J1-20151013_014229_993.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid20387.hprof ...
   [junit4] Heap dump file created [427606152 bytes in 0.832 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 6 suites, 42 tests, 1 error [seed: 9FEAD3AD3FFCFC8E]

Total time: 110 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 983 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/983/

13 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestSolrCoreProperties

Error Message:
ObjectTracker found 6 object(s) that were not released!!! [SolrZkClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [SolrZkClient]
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:236)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.test

Error Message:
KeeperErrorCode = Session expired for /solr

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /solr
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13:EAD986B2CB5717EB]:0)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:294)
	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:291)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:486)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
	at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:90)
	at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.distribSetUp(AbstractDistribZkTestBase.java:72)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.distribSetUp(AbstractFullDistribZkTestBase.java:197)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:933)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest

Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest:     1) Thread[id=9036, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-DeleteLastCustomShardedReplicaTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)    2) Thread[id=9030, name=Thread-6103, state=WAITING, group=TGRP-DeleteLastCustomShardedReplicaTest]         at java.lang.Object.wait(Native Method)         at java.lang.Thread.join(Thread.java:1281)         at java.lang.Thread.join(Thread.java:1355)         at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)    3) Thread[id=9031, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DeleteLastCustomShardedReplicaTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=9032, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteLastCustomShardedReplicaTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest: 
   1) Thread[id=9036, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-DeleteLastCustomShardedReplicaTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
   2) Thread[id=9030, name=Thread-6103, state=WAITING, group=TGRP-DeleteLastCustomShardedReplicaTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Thread.join(Thread.java:1281)
        at java.lang.Thread.join(Thread.java:1355)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
        at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)
        at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)
   3) Thread[id=9031, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DeleteLastCustomShardedReplicaTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=9032, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteLastCustomShardedReplicaTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)


FAILED:  org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection

Error Message:
Delete action failed!

Stack Trace:
java.lang.AssertionError: Delete action failed!
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13:71EE8B0754C4C3B5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.SolrCloudExampleTest.doTestDeleteAction(SolrCloudExampleTest.java:169)
	at org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection(SolrCloudExampleTest.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestCodecSupport

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.apache.lucene.util.TestRuleRestoreSystemProperties.before(TestRuleRestoreSystemProperties.java:53)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.before(TestRuleAdapter.java:26)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:35)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.CSVRequestHandlerTest

Error Message:
ERROR: SolrIndexSearcher opens=1 closes=27

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=1 closes=27
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:467)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:233)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.MoreLikeThisHandlerTest

Error Message:
ERROR: SolrIndexSearcher opens=2 closes=20

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=2 closes=20
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:467)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:233)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.rest.schema.TestManagedSchemaFieldTypeResource

Error Message:
ERROR: SolrIndexSearcher opens=1 closes=28

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=1 closes=28
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:467)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:233)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.schema.TestCloudManagedSchemaConcurrent.test

Error Message:
QUERY FAILED: xpath=/response/lst[@name='responseHeader']/int[@name='status'][.='0']  request=/schema/fieldtypes/newfieldtypePut12?wt=xml&updateTimeoutSecs=15  response=<?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">500</int>   <int name="QTime">15570</int> </lst> <lst name="error">   <str name="msg">6 out of 8 replicas failed to update their schema to version 83 within 15 seconds! Failed cores: [http://127.0.0.1:58910/collection1/, http://127.0.0.1:46483/collection1/, http://127.0.0.1:44303/collection1/, http://127.0.0.1:34415/collection1/, http://127.0.0.1:43603/collection1/, http://127.0.0.1:36992/collection1/]</str>   <str name="trace">org.apache.solr.common.SolrException: 6 out of 8 replicas failed to update their schema to version 83 within 15 seconds! Failed cores: [http://127.0.0.1:58910/collection1/, http://127.0.0.1:46483/collection1/, http://127.0.0.1:44303/collection1/, http://127.0.0.1:34415/collection1/, http://127.0.0.1:43603/collection1/, http://127.0.0.1:36992/collection1/]  at org.apache.solr.schema.ManagedIndexSchema.waitForSchemaZkVersionAgreement(ManagedIndexSchema.java:260)  at org.apache.solr.rest.schema.BaseFieldTypeResource.addNewFieldTypes(BaseFieldTypeResource.java:91)  at org.apache.solr.rest.schema.FieldTypeResource.addOrUpdateFieldType(FieldTypeResource.java:167)  at org.apache.solr.rest.schema.FieldTypeResource.put(FieldTypeResource.java:154)  at org.restlet.resource.ServerResource.doHandle(ServerResource.java:447)  at org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:359)  at org.restlet.resource.ServerResource.handle(ServerResource.java:1044)  at org.restlet.resource.Finder.handle(Finder.java:236)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.routing.Router.doHandle(Router.java:422)  at org.restlet.routing.Router.handle(Router.java:639)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.engine.application.StatusFilter.doHandle(StatusFilter.java:140)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)  at org.restlet.engine.application.ApplicationHelper.handle(ApplicationHelper.java:75)  at org.restlet.Application.handle(Application.java:385)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.routing.Router.doHandle(Router.java:422)  at org.restlet.routing.Router.handle(Router.java:639)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.routing.Router.doHandle(Router.java:422)  at org.restlet.routing.Router.handle(Router.java:639)  at org.restlet.routing.Filter.doHandle(Filter.java:150)  at org.restlet.routing.Filter.handle(Filter.java:197)  at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)  at org.restlet.Component.handle(Component.java:408)  at org.restlet.Server.handle(Server.java:507)  at org.restlet.engine.connector.ServerHelper.handle(ServerHelper.java:63)  at org.restlet.engine.adapter.HttpServerHelper.handle(HttpServerHelper.java:143)  at org.restlet.ext.servlet.ServerServlet.service(ServerServlet.java:1117)  at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)  at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:191)  at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:72)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  at org.eclipse.jetty.server.Server.handle(Server.java:499)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)  at java.lang.Thread.run(Thread.java:745) </str>   <int name="code">500</int> </lst> </response> 

Stack Trace:
java.lang.AssertionError: QUERY FAILED: xpath=/response/lst[@name='responseHeader']/int[@name='status'][.='0']  request=/schema/fieldtypes/newfieldtypePut12?wt=xml&updateTimeoutSecs=15  response=<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">500</int>
  <int name="QTime">15570</int>
</lst>
<lst name="error">
  <str name="msg">6 out of 8 replicas failed to update their schema to version 83 within 15 seconds! Failed cores: [http://127.0.0.1:58910/collection1/, http://127.0.0.1:46483/collection1/, http://127.0.0.1:44303/collection1/, http://127.0.0.1:34415/collection1/, http://127.0.0.1:43603/collection1/, http://127.0.0.1:36992/collection1/]</str>
  <str name="trace">org.apache.solr.common.SolrException: 6 out of 8 replicas failed to update their schema to version 83 within 15 seconds! Failed cores: [http://127.0.0.1:58910/collection1/, http://127.0.0.1:46483/collection1/, http://127.0.0.1:44303/collection1/, http://127.0.0.1:34415/collection1/, http://127.0.0.1:43603/collection1/, http://127.0.0.1:36992/collection1/]
	at org.apache.solr.schema.ManagedIndexSchema.waitForSchemaZkVersionAgreement(ManagedIndexSchema.java:260)
	at org.apache.solr.rest.schema.BaseFieldTypeResource.addNewFieldTypes(BaseFieldTypeResource.java:91)
	at org.apache.solr.rest.schema.FieldTypeResource.addOrUpdateFieldType(FieldTypeResource.java:167)
	at org.apache.solr.rest.schema.FieldTypeResource.put(FieldTypeResource.java:154)
	at org.restlet.resource.ServerResource.doHandle(ServerResource.java:447)
	at org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:359)
	at org.restlet.resource.ServerResource.handle(ServerResource.java:1044)
	at org.restlet.resource.Finder.handle(Finder.java:236)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Router.doHandle(Router.java:422)
	at org.restlet.routing.Router.handle(Router.java:639)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.engine.application.StatusFilter.doHandle(StatusFilter.java:140)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)
	at org.restlet.engine.application.ApplicationHelper.handle(ApplicationHelper.java:75)
	at org.restlet.Application.handle(Application.java:385)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Router.doHandle(Router.java:422)
	at org.restlet.routing.Router.handle(Router.java:639)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Router.doHandle(Router.java:422)
	at org.restlet.routing.Router.handle(Router.java:639)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)
	at org.restlet.Component.handle(Component.java:408)
	at org.restlet.Server.handle(Server.java:507)
	at org.restlet.engine.connector.ServerHelper.handle(ServerHelper.java:63)
	at org.restlet.engine.adapter.HttpServerHelper.handle(HttpServerHelper.java:143)
	at org.restlet.ext.servlet.ServerServlet.service(ServerServlet.java:1117)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:191)
	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:72)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
</str>
  <int name="code">500</int>
</lst>
</response>

	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13:EAD986B2CB5717EB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.verifySuccess(TestCloudManagedSchemaConcurrent.java:107)
	at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.addFieldTypePut(TestCloudManagedSchemaConcurrent.java:162)
	at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.access$600(TestCloudManagedSchemaConcurrent.java:45)
	at org.apache.solr.schema.TestCloudManagedSchemaConcurrent$Operation$6.execute(TestCloudManagedSchemaConcurrent.java:327)
	at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.concurrentOperationsTest(TestCloudManagedSchemaConcurrent.java:450)
	at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.test(TestCloudManagedSchemaConcurrent.java:252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space


FAILED:  junit.framework.TestSuite.org.apache.solr.util.hll.BigEndianAscendingWordSerializerTest

Error Message:
The test or suite printed 362170 bytes to stdout and stderr, even though the limit was set to 8192 bytes. Increase the limit with @Limit, ignore it completely with @SuppressSysoutChecks or run with -Dtests.verbose=true

Stack Trace:
java.lang.AssertionError: The test or suite printed 362170 bytes to stdout and stderr, even though the limit was set to 8192 bytes. Increase the limit with @Limit, ignore it completely with @SuppressSysoutChecks or run with -Dtests.verbose=true
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.apache.lucene.util.TestRuleLimitSysouts.afterIfSuccessful(TestRuleLimitSysouts.java:212)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterIfSuccessful(TestRuleAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:37)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=92125, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=92125, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57759/nagv: collection already exists: awholynewstresscollection_collection5_1
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=19036, name=coreZkRegister-6554-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19036, name=coreZkRegister-6554-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11191 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/init-core-data-001
   [junit4]   2> 3617295 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[628DB96865AB7A13]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 3617298 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3617313 INFO  (Thread-9426) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3617313 INFO  (Thread-9426) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3617410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.ZkTestServer start zk server on port:51025
   [junit4]   2> 3617410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3617434 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3617458 INFO  (zkCallback-1857-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@81185d1 name:ZooKeeperConnection Watcher:127.0.0.1:51025 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3617458 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3617458 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3617458 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3617461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3617470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3617478 INFO  (zkCallback-1858-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@603f1f8e name:ZooKeeperConnection Watcher:127.0.0.1:51025/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3617478 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3617478 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3617478 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3617480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3617481 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3617482 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3617483 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3617483 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3617485 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3617485 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3617487 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3617487 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3617488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3617488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3617489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3617490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3617491 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3617491 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3617492 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3617493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3617494 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3617494 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3617495 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3617495 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3617496 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3617497 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3617498 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3617498 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3617902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1
   [junit4]   2> 3617904 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3617906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ceed70d{/,null,AVAILABLE}
   [junit4]   2> 3617908 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.e.j.s.ServerConnector Started ServerConnector@458741f{HTTP/1.1}{127.0.0.1:48234}
   [junit4]   2> 3617908 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.e.j.s.Server Started @3620790ms
   [junit4]   2> 3617908 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/tempDir-001/control/data, hostPort=48234, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores}
   [junit4]   2> 3617909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 3617909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/'
   [junit4]   2> 3617909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3617909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3617925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3617926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3617946 INFO  (zkCallback-1859-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1384cafd name:ZooKeeperConnection Watcher:127.0.0.1:51025/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3617946 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3617946 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3617947 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3617947 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/solr.xml
   [junit4]   2> 3617970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores
   [junit4]   2> 3617970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.CoreContainer New CoreContainer 827989123
   [junit4]   2> 3617970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/]
   [junit4]   2> 3617970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/lib
   [junit4]   2> 3617971 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/lib).
   [junit4]   2> 3617987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3617989 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3617989 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3617989 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3617989 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51025/solr
   [junit4]   2> 3617989 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3617990 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3618014 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3618034 INFO  (zkCallback-1861-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11f8e7a3 name:ZooKeeperConnection Watcher:127.0.0.1:51025 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3618034 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3618035 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3618054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3618070 INFO  (zkCallback-1862-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1147ff06 name:ZooKeeperConnection Watcher:127.0.0.1:51025/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3618070 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3618071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3618072 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3618075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3618076 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3618077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3618079 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3618080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3618081 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3618082 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3618082 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48234_
   [junit4]   2> 3618083 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48234_
   [junit4]   2> 3618084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3618085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3618086 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3618086 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94673823303925764-127.0.0.1:48234_-n_0000000000
   [junit4]   2> 3618086 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48234_
   [junit4]   2> 3618086 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3618087 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.Overseer Overseer (id=94673823303925764-127.0.0.1:48234_-n_0000000000) starting
   [junit4]   2> 3618089 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3618093 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3618106 INFO  (OverseerStateUpdate-94673823303925764-127.0.0.1:48234_-n_0000000000) [n:127.0.0.1:48234_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3618106 INFO  (OverseerStateUpdate-94673823303925764-127.0.0.1:48234_-n_0000000000) [n:127.0.0.1:48234_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3618114 INFO  (OverseerCollectionConfigSetProcessor-94673823303925764-127.0.0.1:48234_-n_0000000000) [n:127.0.0.1:48234_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3618130 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3618144 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3618144 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3618184 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores
   [junit4]   2> 3618185 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1, collection=control_collection, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, shard=, coreNodeName=, name=collection1, transient=false, config=solrconfig.xml}
   [junit4]   2> 3618185 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/
   [junit4]   2> 3618185 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3618195 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3618195 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3618201 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3618206 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3618206 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[628DB96865AB7A13]) [n:127.0.0.1:48234_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3618207 INFO  (OverseerStateUpdate-94673823303925764-127.0.0.1:48234_-n_0000000000) [n:127.0.0.1:48234_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:48234_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:48234",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 3618207 INFO  (OverseerStateUpdate-94673823303925764-127.0.0.1:48234_-n_0000000000) [n:127.0.0.1:48234_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:48234_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:48234",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3618207 INFO  (OverseerStateUpdate-94673823303925764-127.0.0.1:48234_-n_0000000000) [n:127.0.0.1:48234_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3618208 INFO  (OverseerStateUpdate-94673823303925764-127.0.0.1:48234_-n_0000000000) [n:127.0.0.1:48234_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3618209 INFO  (zkCallback-1862-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3618222 INFO  (zkCallback-1862-thread-2-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3619201 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3619201 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3619203 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3619203 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3619203 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3619203 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/'
   [junit4]   2> 3619203 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3619203 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3619230 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3619236 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3619257 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3619312 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3619317 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3619324 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3619683 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3619692 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3619694 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3619703 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3619706 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3619709 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3619710 WARN  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3619710 WARN  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3619710 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3619710 WARN  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3619711 WARN  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3619712 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3619712 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3619712 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3619712 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3619712 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@448abbbc
   [junit4]   2> 3619713 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/data
   [junit4]   2> 3619713 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3619713 WARN  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3619713 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_628DB96865AB7A13-001/control-001/cores/collection1/data/index
   [junit4]   2> 3619713 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@28009f39
   [junit4]   2> 3619714 INFO  (coreLoadExecutor-6516-thread-1-processing-n:127.0.0.1:48234_) [n:127.0.0.1:48234_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@40d56bb3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7b91d605),segFN=segments_1,generation=1}
   [junit4]   2> 3619714 INFO  (coreLoadExecutor-6516-

[...truncated too long message...]

.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 55493875 WARN  (zkCallback-611-thread-1-SendThread(127.0.0.1:53595)) [    ] o.a.z.ClientCnxn Session 0x1505c21201f0014 for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 55494024 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[628DB96865AB7A13]-SendThread(127.0.0.1:49536)) [n:127.0.0.1:55628_    ] o.a.z.ClientCnxn Session 0x1505929c215000f for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 55495633 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[628DB96865AB7A13]-SendThread(127.0.0.1:49536)) [n:127.0.0.1:55628_    ] o.a.z.ClientCnxn Session 0x1505929c215000f for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 55495847 WARN  (zkCallback-611-thread-1-SendThread(127.0.0.1:53595)) [    ] o.a.z.ClientCnxn Session 0x1505c21201f0014 for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 55497046 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[628DB96865AB7A13]-SendThread(127.0.0.1:49536)) [n:127.0.0.1:55628_    ] o.a.z.ClientCnxn Session 0x1505929c215000f for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 55497064 WARN  (zkCallback-611-thread-1-SendThread(127.0.0.1:53595)) [    ] o.a.z.ClientCnxn Session 0x1505c21201f0014 for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 55498288 ERROR (SUITE-MoreLikeThisHandlerTest-seed#[628DB96865AB7A13]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ERROR: SolrIndexSearcher opens=2 closes=20
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {_version_=PostingsFormat(name=LuceneFixedGap), multiDefault=PostingsFormat(name=Memory doPackFST= true), subword=PostingsFormat(name=LuceneFixedGap), intDefault=PostingsFormat(name=LuceneFixedGap), name=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), foo_ti=PostingsFormat(name=LuceneFixedGap), timestamp=PostingsFormat(name=Memory doPackFST= true), range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=bg_BG, timezone=Asia/Bangkok
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=147835040,total=516423680
   [junit4]   2> NOTE: All tests run in this JVM: [TestOverriddenPrefixQueryForCustomFieldType, TestTolerantSearch, IndexBasedSpellCheckerTest, TestSimpleQParserPlugin, DocumentBuilderTest, RollingRestartTest, TestSerializedLuceneMatchVersion, TestMiniSolrCloudCluster, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DirectSolrSpellCheckerTest, TestRTGBase, TestUpdate, TestRestManager, TestWordDelimiterFilterFactory, ChaosMonkeyNothingIsSafeTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestSolrConfigHandler, TestLFUCache, HLLUtilTest, SaslZkACLProviderTest, CollectionsAPIDistributedZkTest, SolrCmdDistributorTest, TestCodecSupport, DeleteLastCustomShardedReplicaTest, BigEndianAscendingWordSerializerTest, TestSolrCoreProperties, TestManagedSchemaFieldTypeResource, CircularListTest, CSVRequestHandlerTest, TestConfigSets, TestFiltering, TestFieldTypeCollectionResource, TestFastOutputStream, AnalyticsQueryTest, URLClassifyProcessorTest, SolrInfoMBeanTest, TestNamedUpdateProcessors, SolrIndexSplitterTest, DistributedFacetPivotSmallTest, DistanceUnitsTest, TestExactStatsCache, VMParamsZkACLAndCredentialsProvidersTest, TestSolr4Spatial2, DocValuesMissingTest, TestMissingGroups, SimpleCollectionCreateDeleteTest, TestManagedSynonymFilterFactory, HdfsNNFailoverTest, TestReloadDeadlock, SimpleMLTQParserTest, NumericFieldsTest, DOMUtilTest, TermVectorComponentTest, TestFieldCollectionResource, SolrCloudExampleTest, TestCloudManagedSchemaConcurrent, MoreLikeThisHandlerTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MoreLikeThisHandlerTest -Dtests.seed=628DB96865AB7A13 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=bg_BG -Dtests.timezone=Asia/Bangkok -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | MoreLikeThisHandlerTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=2 closes=20
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([628DB96865AB7A13]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:467)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:233)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [509/540] on J1 in 122.59s, 1 test, 1 failure <<< FAILURES!

[...truncated 95 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151011_185759_184.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid20628.hprof ...
   [junit4] Heap dump file created [653927027 bytes in 25.330 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 19 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 540 suites (7 ignored), 2126 tests, 9 suite-level errors, 3 errors, 2 failures, 29 ignored (17 assumptions) [seed: 628DB96865AB7A13]

Total time: 1055 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 982 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/982/

2 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=71755, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=71755, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51491: Could not find collection : awholynewstresscollection_collection1_1
	at __randomizedtesting.SeedInfo.seed([3B9502A82108B459]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=16629, name=coreZkRegister-3500-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16629, name=coreZkRegister-3500-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([3B9502A82108B459]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11366 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/init-core-data-001
   [junit4]   2> 2391948 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[3B9502A82108B459]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /k_/d
   [junit4]   2> 2391953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2391954 INFO  (Thread-10637) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2391954 INFO  (Thread-10637) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2392054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.ZkTestServer start zk server on port:54411
   [junit4]   2> 2392054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2392055 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392058 INFO  (zkCallback-1015-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d95fdf name:ZooKeeperConnection Watcher:127.0.0.1:54411 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2392059 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2392061 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2392062 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392063 INFO  (zkCallback-1016-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@697f73d4 name:ZooKeeperConnection Watcher:127.0.0.1:54411/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392064 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2392064 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2392066 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2392067 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2392069 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2392071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2392071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2392073 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2392074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2392075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2392076 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2392077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2392077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2392079 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2392079 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2392081 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2392081 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2392082 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2392082 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2392084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2392084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2392086 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2392086 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2392087 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2392087 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2392089 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2392089 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2392363 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1
   [junit4]   2> 2392365 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2392367 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@197afa63{/k_/d,null,AVAILABLE}
   [junit4]   2> 2392367 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5b9c29d5{HTTP/1.1}{127.0.0.1:34283}
   [junit4]   2> 2392367 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.Server Started @2395577ms
   [junit4]   2> 2392367 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores, hostPort=34283, hostContext=/k_/d, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/tempDir-001/control/data}
   [junit4]   2> 2392368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6f5e0eab
   [junit4]   2> 2392368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/'
   [junit4]   2> 2392368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2392368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2392383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2392384 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392385 INFO  (zkCallback-1017-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@449d823 name:ZooKeeperConnection Watcher:127.0.0.1:54411/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392386 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2392387 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2392387 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/solr.xml
   [junit4]   2> 2392401 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores
   [junit4]   2> 2392401 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoreContainer New CoreContainer 266071655
   [junit4]   2> 2392401 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/]
   [junit4]   2> 2392401 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/lib
   [junit4]   2> 2392401 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/lib).
   [junit4]   2> 2392409 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2392410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2392411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2392411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2392411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54411/solr
   [junit4]   2> 2392411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2392411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2392411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392413 INFO  (zkCallback-1019-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5053da5f name:ZooKeeperConnection Watcher:127.0.0.1:54411 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392413 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392413 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2392415 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2392416 INFO  (zkCallback-1020-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4078f35b name:ZooKeeperConnection Watcher:127.0.0.1:54411/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2392416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2392417 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2392419 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2392421 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2392423 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2392425 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2392427 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2392429 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2392430 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2392431 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2392432 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34283_k_%2Fd
   [junit4]   2> 2392433 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34283_k_%2Fd
   [junit4]   2> 2392435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2392436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2392437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2392438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000
   [junit4]   2> 2392438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34283_k_%2Fd
   [junit4]   2> 2392438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2392440 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer Overseer (id=94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) starting
   [junit4]   2> 2392441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2392447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2392447 INFO  (OverseerCollectionConfigSetProcessor-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2392447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2392447 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2392463 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2392463 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2392464 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores
   [junit4]   2> 2392464 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, collection=control_collection, transient=false, schema=schema.xml, dataDir=data/, name=collection1, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/, coreNodeName=, loadOnStartup=true, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1}
   [junit4]   2> 2392465 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/
   [junit4]   2> 2392465 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2392466 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2392466 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2392466 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2392466 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2392467 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34283/k_/d",
   [junit4]   2>   "node_name":"127.0.0.1:34283_k_%2Fd",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 2392468 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34283/k_/d",
   [junit4]   2>   "node_name":"127.0.0.1:34283_k_%2Fd",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 2392468 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2392468 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2392468 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2392470 INFO  (zkCallback-1020-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2393468 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2393468 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2393469 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2393469 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2393470 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2393470 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/'
   [junit4]   2> 2393470 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2393470 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2393486 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2393492 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2393505 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2393548 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2393549 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2393555 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2393795 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2393803 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2393805 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2393814 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2393817 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2393819 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2393820 WARN  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2393820 WARN  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2393820 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2393821 WARN  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2393821 WARN  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2393822 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2393822 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2393822 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2393822 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2393823 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c4d0a21
   [junit4]   2> 2393823 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/data
   [junit4]   2> 2393823 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2393823 WARN  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2393823 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/data/index
   [junit4]   2> 2393824 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@708ca0e7
   [junit4]   2> 2393824 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2097d76b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@50bfc284),segFN=segments_1,generation=1}
   [junit4]   2> 2393824 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2393825 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/data/
   [junit4]   2> 2393825 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2393828 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2393829 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2393829 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2393829 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2393829 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2393829 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2393830 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2393830 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2393830 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2393832 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2393833 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2393835 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2393836 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2393839 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/admin/segments,/admin/mbeans,/update/json/docs,/get,/update/csv,/admin/logging,/update,/admin/file,/admin/ping,/admin/properties,standard,/admin/system,/admin/threads,/replication,/update/json,/config,/admin/luke,/schema
   [junit4]   2> 2393841 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2393841 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2393841 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2393842 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2393842 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2393842 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@455bb49c
   [junit4]   2> 2393843 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2097d76b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@50bfc284),segFN=segments_1,generation=1}
   [junit4]   2> 2393843 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2393843 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@579e99b3[collection1] main
   [junit4]   2> 2393843 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2393844 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2393844 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2393844 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2393844 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2393844 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2393845 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2393845 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2393845 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2393845 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2393845 INFO  (searcherExecutor-3463-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@579e99b3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2393846 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2393846 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2393846 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2393846 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514688741624512512
   [junit4]   2> 2393848 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514688741624512512
   [junit4]   2> 2393848 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2393848 INFO  (coreLoadExecutor-3462-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2393849 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34283/k_/d collection:control_collection shard:shard1
   [junit4]   2> 2393849 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2393854 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94668046227603460-core_node1-n_0000000000
   [junit4]   2> 2393855 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2393857 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2393857 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2393857 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34283/k_/d/collection1/
   [junit4]   2> 2393857 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2393857 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34283/k_/d/collection1/ has no replicas
   [junit4]   2> 2393857 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34283/k_/d/collection1/ shard1
   [junit4]   2> 2393857 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2393858 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2393860 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2393862 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:34283/k_/d",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2393965 INFO  (zkCallback-1020-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2393967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2393968 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2393969 INFO  (zkCallback-1022-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4e7be7e name:ZooKeeperConnection Watcher:127.0.0.1:54411/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2393969 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2393970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2393970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2393972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2393972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2393972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2393972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2393973 INFO  (zkCallback-1023-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b6f60b name:ZooKeeperConnection Watcher:127.0.0.1:54411/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2393973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2393973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2393975 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 2393975 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2394013 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:34283/k_/d/collection1/ and leader is http://127.0.0.1:34283/k_/d/collection1/
   [junit4]   2> 2394013 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34283/k_/d
   [junit4]   2> 2394013 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2394014 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2394014 INFO  (coreZkRegister-3456-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:34283_k_%2Fd x:collection1 c:control_collection) [n:127.0.0.1:34283_k_%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2394015 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34283/k_/d",
   [junit4]   2>   "node_name":"127.0.0.1:34283_k_%2Fd",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection"} current state version: 2
   [junit4]   2> 2394016 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34283/k_/d",
   [junit4]   2>   "node_name":"127.0.0.1:34283_k_%2Fd",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 2394016 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2394120 INFO  (zkCallback-1022-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2394120 INFO  (zkCallback-1020-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2394251 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/cores/collection1
   [junit4]   2> 2394252 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001
   [junit4]   2> 2394253 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2394254 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@781256ff{/k_/d,null,AVAILABLE}
   [junit4]   2> 2394255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.ServerConnector Started ServerConnector@69eeece2{HTTP/1.1}{127.0.0.1:47752}
   [junit4]   2> 2394255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.e.j.s.Server Started @2397465ms
   [junit4]   2> 2394255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=47752, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/k_/d}
   [junit4]   2> 2394255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6f5e0eab
   [junit4]   2> 2394255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/'
   [junit4]   2> 2394255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2394255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2394271 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2394271 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2394272 INFO  (zkCallback-1024-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18a68cf9 name:ZooKeeperConnection Watcher:127.0.0.1:54411/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2394272 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2394273 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2394274 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2394274 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/solr.xml
   [junit4]   2> 2394287 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/cores
   [junit4]   2> 2394288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoreContainer New CoreContainer 1069611092
   [junit4]   2> 2394288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/]
   [junit4]   2> 2394288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/lib
   [junit4]   2> 2394288 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/lib).
   [junit4]   2> 2394296 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2394297 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2394297 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2394297 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2394298 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54411/solr
   [junit4]   2> 2394298 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2394298 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2394299 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2394300 INFO  (zkCallback-1026-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6aa4181e name:ZooKeeperConnection Watcher:127.0.0.1:54411 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2394300 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2394300 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2394301 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2394302 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62f2bb0e name:ZooKeeperConnection Watcher:127.0.0.1:54411/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2394303 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2394307 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2395313 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47752_k_%2Fd
   [junit4]   2> 2395313 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47752_k_%2Fd
   [junit4]   2> 2395314 INFO  (zkCallback-1020-thread-1-processing-n:127.0.0.1:34283_k_%2Fd) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2395314 INFO  (zkCallback-1022-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2395315 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2395317 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2395318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94668046227603465-127.0.0.1:47752_k_%2Fd-n_0000000001
   [junit4]   2> 2395318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000 to know if I could be the leader
   [junit4]   2> 2395333 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2395333 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2395334 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/cores
   [junit4]   2> 2395335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/cores/collection1, dataDir=data/, config=solrconfig.xml, loadOnStartup=true, shard=, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/cores/collection1/, name=collection1, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, coreNodeName=}
   [junit4]   2> 2395335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_3B9502A82108B459-001/shard-1-001/cores/collection1/
   [junit4]   2> 2395335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2395337 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2395337 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[3B9502A82108B459]) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2395338 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2395338 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2395339 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2395339 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 2395340 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:47752/k_/d",
   [junit4]   2>   "node_name":"127.0.0.1:47752_k_%2Fd",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1"} current state version: 3
   [junit4]   2> 2395340 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 2395340 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2395340 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:47752/k_/d",
   [junit4]   2>   "node_name":"127.0.0.1:47752_k_%2Fd",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1"}
   [junit4]   2> 2395340 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 2395341 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2395443 INFO  (OverseerStateUpdate-94668046227603460-127.0.0.1:34283_k_%2Fd-n_0000000000) [n:127.0.0.1:34283_k_%2Fd    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 2395444 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2395445 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 2396340 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2396340 INFO  (coreLoadExecutor-3473-thread-1-processing-n:127.0.0.1:47752_k_%2Fd) [n:127.0.0.1:47752_k_%2Fd c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2396341 INFO  (coreLoadExecutor-3473-thread-1-proces

[...truncated too long message...]

MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3528173 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[3B9502A82108B459]-worker) [n:127.0.0.1:44705_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Oct 10, 2015 7:48:05 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {a_i1=BlockTreeOrds(blocksize=128), _version_=BlockTreeOrds(blocksize=128), range_facet_l_dv=PostingsFormat(name=MockRandom), a_t=BlockTreeOrds(blocksize=128), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), multiDefault=PostingsFormat(name=Direct), id=PostingsFormat(name=MockRandom), other_tl1=BlockTreeOrds(blocksize=128), intDefault=BlockTreeOrds(blocksize=128), timestamp=BlockTreeOrds(blocksize=128), range_facet_l=BlockTreeOrds(blocksize=128), range_facet_i_dv=BlockTreeOrds(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=ar_BH, timezone=Asia/Ashkhabad
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=82148408,total=524288000
   [junit4]   2> NOTE: All tests run in this JVM: [TestSchemaNameResource, TestClusterStateMutator, DistributedDebugComponentTest, DocumentAnalysisRequestHandlerTest, JsonLoaderTest, BlockCacheTest, HLLSerializationTest, BasicZkTest, AnalysisAfterCoreReloadTest, HdfsUnloadDistributedZkTest, FileUtilsTest, SharedFSAutoReplicaFailoverUtilsTest, TestCloudPivotFacet, OverseerStatusTest, TestComplexPhraseQParserPlugin, TestSchemaSimilarityResource, PathHierarchyTokenizerFactoryTest, SignatureUpdateProcessorFactoryTest, IndexSchemaTest, TestSolr4Spatial, TestConfigReload, TestMacros, ExternalFileFieldSortTest, SpellingQueryConverterTest, TestAuthorizationFramework, TestMiniSolrCloudClusterSSL, FileBasedSpellCheckerTest, TestLuceneMatchVersion, XsltUpdateRequestHandlerTest, CopyFieldTest, TestDistributedSearch, TriLevelCompositeIdRoutingTest, CollectionTooManyReplicasTest, PingRequestHandlerTest, TestSolrCLIRunExample, TestCollationFieldDocValues, TestReplicationHandler, CustomCollectionTest, TestRemoteStreaming, TestFoldingMultitermQuery, RemoteQueryErrorTest, TestRangeQuery, DistributedMLTComponentTest, UUIDUpdateProcessorFallbackTest, SOLR749Test, PolyFieldTest, TestConfig, TestFaceting, TestRawResponseWriter, ResponseLogComponentTest, TestOmitPositions, SuggestComponentTest, OverseerCollectionConfigSetProcessorTest, TestOrdValues, TestCSVLoader, DistributedFacetPivotWhiteBoxTest, TestFreeTextSuggestions, SolrXmlInZkTest, TestArbitraryIndexDir, FullHLLTest, StandardRequestHandlerTest, DefaultValueUpdateProcessorTest, DistributedQueryComponentOptimizationTest, MultiThreadedOCPTest, RecoveryAfterSoftCommitTest, ExplicitHLLTest, TestManagedSchemaDynamicFieldResource, TestTrackingShardHandlerFactory, ReplicationFactorTest, DistribJoinFromCollectionTest, TestFunctionQuery, TestMaxScoreQueryParser, EchoParamsTest, TestBulkSchemaConcurrent, TestManagedSchema, TestSolrConfigHandlerConcurrent, BlockDirectoryTest, TimeZoneUtilsTest, TestLMDirichletSimilarityFactory, AliasIntegrationTest, DistributedFacetPivotLargeTest, DataDrivenBlockJoinTest, CoreMergeIndexesAdminHandlerTest, TestBadConfig, BigEndianAscendingWordDeserializerTest, HdfsDirectoryTest, ParsingFieldUpdateProcessorsTest, TestTolerantSearch, AssignTest, AsyncMigrateRouteKeyTest, ConcurrentDeleteAndCreateCollectionTest, LeaderFailoverAfterPartitionTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerRolesTest, SSLMigrationTest, ShardSplitTest, TestAuthenticationFramework, TestCloudInspectUtil, TestCollectionAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestLeaderElectionZkExpiry, TestLeaderInitiatedRecoveryThread, TestRandomRequestDistribution, TestShortCircuitedRequests, HdfsBasicDistributedZk2Test, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [463/540] on J0 in 371.15s, 1 test, 1 error <<< FAILURES!

[...truncated 260 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 540 suites (7 ignored), 2131 tests, 2 errors, 29 ignored (17 assumptions) [seed: 3B9502A82108B459]

Total time: 187 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 981 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/981/

3 tests failed.
FAILED:  org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection

Error Message:
Delete action failed!

Stack Trace:
java.lang.AssertionError: Delete action failed!
	at __randomizedtesting.SeedInfo.seed([FD8F3F9C919320A6:EEEC0DF3A0FC9900]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.SolrCloudExampleTest.doTestDeleteAction(SolrCloudExampleTest.java:169)
	at org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection(SolrCloudExampleTest.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2770, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2770, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39407: Could not find collection : awholynewstresscollection_collection1_1
	at __randomizedtesting.SeedInfo.seed([FD8F3F9C919320A6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=67647, name=coreZkRegister-4391-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=67647, name=coreZkRegister-4391-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([FD8F3F9C919320A6]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10476 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/init-core-data-001
   [junit4]   2> 877612 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[FD8F3F9C919320A6]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 877612 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[FD8F3F9C919320A6]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 877616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 877617 INFO  (Thread-1199) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 877617 INFO  (Thread-1199) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 877717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.ZkTestServer start zk server on port:55231
   [junit4]   2> 877717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 877723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 877745 INFO  (zkCallback-183-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62281cbd name:ZooKeeperConnection Watcher:127.0.0.1:55231 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 877745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 877745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 877745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 877750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 877762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 877766 INFO  (zkCallback-184-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7ec3566a name:ZooKeeperConnection Watcher:127.0.0.1:55231/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 877766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 877766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 877766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 877768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 877770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 877771 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 877773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 877773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 877776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 877776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 877778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 877778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 877780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 877780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 877782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 877782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 877784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 877784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 877786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 877786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 877788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 877788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 877790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 877790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 877792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 877792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 877794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 877794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 878139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1
   [junit4]   2> 878158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 878191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70f72e42{/,null,AVAILABLE}
   [junit4]   2> 878202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.ServerConnector Started ServerConnector@73fe28fe{HTTP/1.1}{127.0.0.1:51007}
   [junit4]   2> 878202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.Server Started @881245ms
   [junit4]   2> 878202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores, hostPort=51007, hostContext=/}
   [junit4]   2> 878202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 878203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/'
   [junit4]   2> 878203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 878203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 878224 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 878230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 878237 INFO  (zkCallback-185-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b3c745c name:ZooKeeperConnection Watcher:127.0.0.1:55231/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 878238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 878238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 878241 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 878241 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/solr.xml
   [junit4]   2> 878266 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores
   [junit4]   2> 878267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoreContainer New CoreContainer 1721211612
   [junit4]   2> 878267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/]
   [junit4]   2> 878267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/lib
   [junit4]   2> 878267 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/lib).
   [junit4]   2> 878282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 878284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 878284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 878284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 878284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55231/solr
   [junit4]   2> 878284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 878284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 878292 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 878294 INFO  (zkCallback-187-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@155c7d1e name:ZooKeeperConnection Watcher:127.0.0.1:55231 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 878294 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 878294 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 878310 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 878315 INFO  (zkCallback-188-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@509eaeb0 name:ZooKeeperConnection Watcher:127.0.0.1:55231/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 878315 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 878316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 878318 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 878321 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 878323 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 878325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 878327 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 878328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 878329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 878330 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 878331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51007_
   [junit4]   2> 878332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51007_
   [junit4]   2> 878333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 878334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 878335 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 878336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94662258141429764-127.0.0.1:51007_-n_0000000000
   [junit4]   2> 878336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51007_
   [junit4]   2> 878336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 878337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer Overseer (id=94662258141429764-127.0.0.1:51007_-n_0000000000) starting
   [junit4]   2> 878339 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 878345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 878346 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 878347 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 878353 INFO  (OverseerCollectionConfigSetProcessor-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 878361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 878377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 878377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 878378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores
   [junit4]   2> 878379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, configSetProperties=configsetprops.json, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1, shard=, dataDir=data/, config=solrconfig.xml, schema=schema.xml, name=collection1, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/, coreNodeName=}
   [junit4]   2> 878379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/
   [junit4]   2> 878379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 878382 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 878382 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 878386 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 878386 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:51007_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 878386 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51007",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51007_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 878387 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51007",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51007_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 878387 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 878387 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 878388 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 878391 INFO  (zkCallback-188-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 878392 INFO  (zkCallback-188-thread-2-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 879387 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 879388 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 879388 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 879388 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 879389 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 879389 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/'
   [junit4]   2> 879389 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 879389 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 879405 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 879421 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 879436 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 879467 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 879468 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 879475 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 880000 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 880011 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 880013 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 880027 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 880031 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 880035 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 880036 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 880037 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 880037 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 880037 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 880037 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 880038 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 880039 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 880039 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 880039 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 880039 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c7f97ce
   [junit4]   2> 880040 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/data
   [junit4]   2> 880040 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/data/index/
   [junit4]   2> 880040 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 880041 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/data/index
   [junit4]   2> 880042 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1523303849, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 880045 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a404804 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@62381447),segFN=segments_1,generation=1}
   [junit4]   2> 880045 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 880045 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/data/
   [junit4]   2> 880045 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 880053 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 880053 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 880053 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 880053 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 880053 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 880053 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 880054 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 880054 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 880054 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 880058 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 880060 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 880061 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 880062 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 880066 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /get,/admin/threads,/replication,/update/csv,/schema,/update/json/docs,/admin/segments,/admin/plugins,/admin/logging,/admin/system,/update,/admin/properties,/admin/mbeans,/admin/file,/config,/admin/ping,standard,/admin/luke,/update/json
   [junit4]   2> 880068 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 880068 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 880068 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 880069 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 880069 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 880070 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=26.0927734375, floorSegmentMB=1.6708984375, forceMergeDeletesPctAllowed=27.3347841271099, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.30546056196163796
   [junit4]   2> 880070 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a404804 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@62381447),segFN=segments_1,generation=1}
   [junit4]   2> 880070 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 880070 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@35ed8a30[collection1] main
   [junit4]   2> 880070 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 880071 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 880071 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 880072 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 880072 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 880072 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 880072 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 880072 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 880073 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 880073 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 880073 INFO  (searcherExecutor-1032-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@35ed8a30[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 880073 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 880074 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 880074 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 880074 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514596132838178816
   [junit4]   2> 880076 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514596132838178816
   [junit4]   2> 880076 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 880077 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 880077 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:51007 collection:control_collection shard:shard1
   [junit4]   2> 880078 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 880087 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94662258141429764-core_node1-n_0000000000
   [junit4]   2> 880088 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 880091 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 880091 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 880091 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 880091 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51007/collection1/
   [junit4]   2> 880091 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 880091 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51007/collection1/ has no replicas
   [junit4]   2> 880091 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51007/collection1/ shard1
   [junit4]   2> 880092 INFO  (zkCallback-188-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 880092 INFO  (zkCallback-188-thread-2-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 880094 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 880097 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 880100 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51007",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 880207 INFO  (zkCallback-188-thread-2-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 880207 INFO  (zkCallback-188-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 880252 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:51007/collection1/ and leader is http://127.0.0.1:51007/collection1/
   [junit4]   2> 880252 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51007
   [junit4]   2> 880252 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 880252 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 880252 INFO  (coreZkRegister-1025-thread-1-processing-x:collection1 n:127.0.0.1:51007_ s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:51007_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 880255 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51007",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51007_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 880256 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51007",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51007_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 880358 INFO  (zkCallback-188-thread-2-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 880358 INFO  (zkCallback-188-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 880389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 880390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 880393 INFO  (zkCallback-190-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a7e564b name:ZooKeeperConnection Watcher:127.0.0.1:55231/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 880393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 880393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 880393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 880397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 880397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 880397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 880398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 880399 INFO  (zkCallback-191-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f877f41 name:ZooKeeperConnection Watcher:127.0.0.1:55231/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 880399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 880399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 880404 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 880404 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 880511 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 880725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores/collection1
   [junit4]   2> 880726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001
   [junit4]   2> 880727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 880730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20940707{/,null,AVAILABLE}
   [junit4]   2> 880730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.ServerConnector Started ServerConnector@9153af4{HTTP/1.1}{127.0.0.1:44911}
   [junit4]   2> 880730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.Server Started @883773ms
   [junit4]   2> 880730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores, hostContext=/, hostPort=44911, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/tempDir-001/jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 880731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 880731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/'
   [junit4]   2> 880731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 880731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 880754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 880756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 880765 INFO  (zkCallback-192-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55e3ec2 name:ZooKeeperConnection Watcher:127.0.0.1:55231/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 880765 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 880765 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 880768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 880768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/solr.xml
   [junit4]   2> 880783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores
   [junit4]   2> 880784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoreContainer New CoreContainer 887351633
   [junit4]   2> 880784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/]
   [junit4]   2> 880784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/lib
   [junit4]   2> 880784 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/lib).
   [junit4]   2> 880792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 880794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 880794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 880794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 880794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55231/solr
   [junit4]   2> 880794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 880795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 880795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 880797 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@89799d2 name:ZooKeeperConnection Watcher:127.0.0.1:55231 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 880797 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 880798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 880802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 880804 INFO  (zkCallback-195-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29360df7 name:ZooKeeperConnection Watcher:127.0.0.1:55231/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 880804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 880812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 881819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44911_
   [junit4]   2> 881820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44911_
   [junit4]   2> 881822 INFO  (zkCallback-190-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 881822 INFO  (zkCallback-188-thread-1-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 881822 INFO  (zkCallback-195-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 881822 INFO  (zkCallback-188-thread-2-processing-n:127.0.0.1:51007_) [n:127.0.0.1:51007_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 881825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 881826 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94662258141429769-127.0.0.1:44911_-n_0000000001
   [junit4]   2> 881827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94662258141429764-127.0.0.1:51007_-n_0000000000 to know if I could be the leader
   [junit4]   2> 881851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 881851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 881852 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores
   [junit4]   2> 881853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, collection=collection1, config=solrconfig.xml, dataDir=data/, loadOnStartup=true, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores/collection1/, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores/collection1, coreNodeName=, transient=false, schema=schema.xml}
   [junit4]   2> 881853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores/collection1/
   [junit4]   2> 881853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 881857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 881857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:44911_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 881857 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 881857 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 881859 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 881859 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 881859 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:44911",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44911_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1"} current state version: 4
   [junit4]   2> 881859 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 881859 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:44911",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44911_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1"}
   [junit4]   2> 881859 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 881860 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 881860 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 881962 INFO  (OverseerStateUpdate-94662258141429764-127.0.0.1:51007_-n_0000000000) [n:127.0.0.1:51007_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 881963 INFO  (zkCallback-195-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 881964 INFO  (zkCallback-195-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 882860 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 882860 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 882860 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 882860 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 882861 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 882861 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD8F3F9C919320A6-001/shard-1-001/cores/collection1/'
   [junit4]   2> 882861 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 882862 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 882879 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 882885 INFO  (coreLoadExecutor-1042-thread-1-processing-n:127.0.0.1:44911_) [n:127.0.0.1:44911_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2

[...truncated too long message...]

766952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@95593ac{HTTP/1.1}{127.0.0.1:37208}
   [junit4]   2> 2766952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FD8F3F9C919320A6]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@46267ee7{/,null,UNAVAILABLE}
   [junit4]   2> 2766954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[FD8F3F9C919320A6]) [n:127.0.0.1:37208_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36935 36935
   [junit4]   2> 2767078 INFO  (Thread-58578) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36935 36935
   [junit4]   2> 2767079 WARN  (Thread-58578) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/aliases.json
   [junit4]   2> 	6	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	35	/solr/clusterstate.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94662358046998532-127.0.0.1:42116_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/live_nodes
   [junit4]   2> 	8	/solr/collections
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=FD8F3F9C919320A6 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_LB -Dtests.timezone=Africa/Bissau -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR    365s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=67647, name=coreZkRegister-4391-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([FD8F3F9C919320A6]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2767094 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[FD8F3F9C919320A6]-worker) [n:127.0.0.1:37208_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_FD8F3F9C919320A6-001
   [junit4]   2> Oct 09, 2015 7:28:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=ar_LB, timezone=Africa/Bissau
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=341330552,total=478674944
   [junit4]   2> NOTE: All tests run in this JVM: [UpdateParamsTest, TestAnalyzeInfixSuggestions, HdfsLockFactoryTest, DirectUpdateHandlerOptimizeTest, TestCollationField, SuggesterWFSTTest, CloudMLTQParserTest, TestJsonFacets, QueryResultKeyTest, OverseerTest, FieldAnalysisRequestHandlerTest, TestReplicationHandlerBackup, ZkNodePropsTest, ChangedSchemaMergeTest, SparseHLLTest, DateRangeFieldTest, RecoveryZkTest, SolrCoreCheckLockOnStartupTest, TestExactSharedStatsCache, TestSort, BadComponentTest, DirectUpdateHandlerTest, TestSolrQueryParserResource, TestSolrQueryParser, ActionThrottleTest, TestFuzzyAnalyzedSuggestions, DistribDocExpirationUpdateProcessorTest, RulesTest, TestShardHandlerFactory, TestNonDefinedSimilarityFactory, ShowFileRequestHandlerTest, AsyncCallRequestStatusResponseTest, TestMinMaxOnMultiValuedField, TestJettySolrRunner, TestFastLRUCache, TestCharFilters, TestDistributedStatsComponentCardinality, TestShortCircuitedRequests, ShardSplitTest, DebugComponentTest, DistributedSuggestComponentTest, TestCopyFieldCollectionResource, TestFastWriter, TestStressReorder, SpatialRPTFieldTypeTest, TermVectorComponentDistributedTest, TestSolrConfigHandlerCloud, TestSimpleTrackingShardHandler, DocExpirationUpdateProcessorFactoryTest, SolrTestCaseJ4Test, TestNoOpRegenerator, TestIndexSearcher, DocValuesMultiTest, TestExtendedDismaxParser, SystemInfoHandlerTest, BitVectorTest, TestStressRecovery, TestLeaderInitiatedRecoveryThread, TestMiniSolrCloudClusterBase, AutoCommitTest, PrimitiveFieldTypeTest, TestJsonRequest, SecurityConfHandlerTest, TestReRankQParserPlugin, TestRemoveLastDynamicCopyField, TestSearchPerf, RAMDirectoryFactoryTest, TestHashQParserPlugin, TestManagedSchemaFieldResource, FullSolrCloudDistribCmdsTest, CursorMarkTest, TestLRUCache, AddBlockUpdateTest, FieldMutatingUpdateProcessorTest, MergeStrategyTest, BadIndexSchemaTest, FacetPivotSmallTest, TestXIncludeConfig, TestCSVResponseWriter, LeaderFailoverAfterPartitionTest, ZkStateWriterTest, TestElisionMultitermQuery, DirectSolrConnectionTest, LeaderElectionTest, TestReversedWildcardFilterFactory, TestFieldTypeResource, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, ZkSolrClientTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedSearch, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, DOMUtilTest, ClusterStateTest, TestUtils, TestDocumentBuilder, UUIDFieldTest, DistributedMLTComponentTest, CursorPagingTest, DistributedIntervalFacetingTest, DeleteReplicaTest, DeleteShardTest, DistributedQueueTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [416/540] on J2 in 365.79s, 1 test, 1 error <<< FAILURES!

[...truncated 408 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 540 suites (7 ignored), 2131 tests, 2 errors, 1 failure, 29 ignored (17 assumptions) [seed: FD8F3F9C919320A6]

Total time: 188 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 980 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/980/

4 tests failed.
FAILED:  org.apache.solr.handler.component.TestDistributedStatsComponentCardinality.test

Error Message:
Captured an uncaught exception in thread: Thread[id=114471, name=Thread-100086, state=RUNNABLE, group=TGRP-TestDistributedStatsComponentCardinality]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=114471, name=Thread-100086, state=RUNNABLE, group=TGRP-TestDistributedStatsComponentCardinality]
	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168:2AF8D351F7608C90]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35068//collection1: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:57228//collection1
	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:624)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=6025, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=6025, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46630: Could not find collection : awholynewstresscollection_collection2_2
	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12513, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12513, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168:2AF8D351F7608C90]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50088/ol_wc/lg: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=48223, name=coreZkRegister-3107-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=48223, name=coreZkRegister-3107-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10230 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/init-core-data-001
   [junit4]   2> 788611 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A2ACEC8B599CE168]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 788611 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A2ACEC8B599CE168]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 788621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 788621 INFO  (Thread-5161) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 788621 INFO  (Thread-5161) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 788721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.ZkTestServer start zk server on port:33713
   [junit4]   2> 788722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 788722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 788726 INFO  (zkCallback-53-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69c82940 name:ZooKeeperConnection Watcher:127.0.0.1:33713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 788727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 788727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 788727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 788730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 788731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 788732 INFO  (zkCallback-54-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74947516 name:ZooKeeperConnection Watcher:127.0.0.1:33713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 788732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 788732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 788732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 788736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 788738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 788739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 788741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 788742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 788745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 788746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 788748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 788748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 788750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 788750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 788753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 788753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 788756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 788756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 788758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 788759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 788761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 788761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 788763 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 788764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 788766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 788766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 788768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 788768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 789118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1
   [junit4]   2> 789121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 789123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7523a197{/,null,AVAILABLE}
   [junit4]   2> 789124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6cc9d6dc{HTTP/1.1}{127.0.0.1:34863}
   [junit4]   2> 789124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.Server Started @792403ms
   [junit4]   2> 789124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/tempDir-001/control/data, hostContext=/, hostPort=34863, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores}
   [junit4]   2> 789125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7f423820
   [junit4]   2> 789125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/'
   [junit4]   2> 789142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 789143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789145 INFO  (zkCallback-55-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c9b7aa5 name:ZooKeeperConnection Watcher:127.0.0.1:33713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 789147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 789147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/solr.xml
   [junit4]   2> 789164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores
   [junit4]   2> 789164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoreContainer New CoreContainer 1494761916
   [junit4]   2> 789164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/]
   [junit4]   2> 789164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/lib
   [junit4]   2> 789164 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/lib).
   [junit4]   2> 789174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 789176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 789176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 789176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 789176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33713/solr
   [junit4]   2> 789176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 789177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 789177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789195 INFO  (zkCallback-57-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72557c4 name:ZooKeeperConnection Watcher:127.0.0.1:33713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 789199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789200 INFO  (zkCallback-58-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25c0e94 name:ZooKeeperConnection Watcher:127.0.0.1:33713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 789204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 789208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 789211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 789213 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 789216 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 789217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 789219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 789221 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 789222 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34863_
   [junit4]   2> 789222 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34863_
   [junit4]   2> 789224 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 789227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 789229 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 789230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94656633793609732-127.0.0.1:34863_-n_0000000000
   [junit4]   2> 789230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34863_
   [junit4]   2> 789230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 789232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer Overseer (id=94656633793609732-127.0.0.1:34863_-n_0000000000) starting
   [junit4]   2> 789234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 789245 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 789245 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 789245 INFO  (OverseerCollectionConfigSetProcessor-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 789246 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 789287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 789287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 789288 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores
   [junit4]   2> 789289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/, schema=schema.xml, dataDir=data/, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1, transient=false, shard=, coreNodeName=, name=collection1, loadOnStartup=true, collection=control_collection, config=solrconfig.xml}
   [junit4]   2> 789289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/
   [junit4]   2> 789289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 789290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 789291 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:34863_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 789291 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 789291 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 789293 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34863",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34863_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 789294 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34863",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34863_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 789294 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 789295 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 789296 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 789298 INFO  (zkCallback-58-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 790295 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 790295 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 790296 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 790296 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 790296 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 790296 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/'
   [junit4]   2> 790314 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 790321 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 790337 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 790371 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 790373 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 790382 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 790672 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 790682 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 790684 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 790703 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 790707 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 790712 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 790714 WARN  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 790714 WARN  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 790714 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 790716 WARN  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 790716 WARN  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 790717 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 790717 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 790717 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 790717 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 790717 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d3c6a55
   [junit4]   2> 790718 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/data
   [junit4]   2> 790719 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/data/index/
   [junit4]   2> 790719 WARN  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 790719 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/data/index
   [junit4]   2> 790719 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=50.4580078125, floorSegmentMB=1.0947265625, forceMergeDeletesPctAllowed=11.25367886223398, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 790720 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@1f2590c6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5b43c723),segFN=segments_1,generation=1}
   [junit4]   2> 790720 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 790721 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/data/
   [junit4]   2> 790721 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 790726 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 790727 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 790727 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 790727 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 790727 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 790728 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 790728 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 790729 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 790729 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 790732 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 790733 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 790735 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 790736 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 790740 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/update/json,/update/csv,/admin/file,/config,/get,/admin/ping,/admin/plugins,/replication,/update/json/docs,standard,/schema,/admin/luke,/update,/admin/threads,/admin/properties,/admin/mbeans,/admin/system,/admin/logging
   [junit4]   2> 790741 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 790742 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 790742 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 790743 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 790743 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 790743 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=2, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=73.5283203125, floorSegmentMB=0.716796875, forceMergeDeletesPctAllowed=24.56174515396994, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 790744 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@1f2590c6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5b43c723),segFN=segments_1,generation=1}
   [junit4]   2> 790744 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 790744 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5fe29281[collection1] main
   [junit4]   2> 790744 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 790745 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 790745 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 790746 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 790746 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 790746 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 790746 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 790746 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 790746 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 790747 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 790747 INFO  (searcherExecutor-365-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5fe29281[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 790747 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 790747 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 790747 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 790748 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514506142925979648
   [junit4]   2> 790750 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514506142925979648
   [junit4]   2> 790750 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 790750 INFO  (coreLoadExecutor-364-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 790751 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34863 collection:control_collection shard:shard1
   [junit4]   2> 790752 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 790756 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94656633793609732-core_node1-n_0000000000
   [junit4]   2> 790757 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 790759 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 790759 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 790759 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34863/collection1/
   [junit4]   2> 790759 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 790759 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34863/collection1/ has no replicas
   [junit4]   2> 790759 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34863/collection1/ shard1
   [junit4]   2> 790760 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 790760 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 790764 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 790768 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:34863",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 790792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 790793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 790794 INFO  (zkCallback-60-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4807b241 name:ZooKeeperConnection Watcher:127.0.0.1:33713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 790795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 790795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 790795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 790798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 790798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 790798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 790799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 790800 INFO  (zkCallback-61-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@354f4b15 name:ZooKeeperConnection Watcher:127.0.0.1:33713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 790800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 790800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 790807 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 790807 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 790808 INFO  (zkCallback-58-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 790808 INFO  (zkCallback-60-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 790818 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:34863/collection1/ and leader is http://127.0.0.1:34863/collection1/
   [junit4]   2> 790818 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34863
   [junit4]   2> 790818 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 790818 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 790818 INFO  (coreZkRegister-358-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:34863_ c:control_collection r:core_node1) [n:127.0.0.1:34863_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 790821 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34863",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34863_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2"} current state version: 2
   [junit4]   2> 790822 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34863",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34863_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 790822 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 790929 INFO  (zkCallback-60-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 790929 INFO  (zkCallback-58-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 791114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores/collection1
   [junit4]   2> 791114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001
   [junit4]   2> 791115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 791117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4abb71e7{/,null,AVAILABLE}
   [junit4]   2> 791118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3331280{HTTP/1.1}{127.0.0.1:53612}
   [junit4]   2> 791118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.e.j.s.Server Started @794397ms
   [junit4]   2> 791118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/tempDir-001/jetty1, hostContext=/, hostPort=53612, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores}
   [junit4]   2> 791119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7f423820
   [junit4]   2> 791119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/'
   [junit4]   2> 791135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 791136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 791137 INFO  (zkCallback-62-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1d703594 name:ZooKeeperConnection Watcher:127.0.0.1:33713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 791137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 791138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 791140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 791140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/solr.xml
   [junit4]   2> 791156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores
   [junit4]   2> 791156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoreContainer New CoreContainer 1717138323
   [junit4]   2> 791157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/]
   [junit4]   2> 791157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/lib
   [junit4]   2> 791157 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/lib).
   [junit4]   2> 791165 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 791166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 791167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 791167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 791167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33713/solr
   [junit4]   2> 791167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 791167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 791168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 791170 INFO  (zkCallback-64-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a0f4a0c name:ZooKeeperConnection Watcher:127.0.0.1:33713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 791170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 791170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 791174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 791175 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8163eeb name:ZooKeeperConnection Watcher:127.0.0.1:33713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 791175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 791185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 792196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53612_
   [junit4]   2> 792197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53612_
   [junit4]   2> 792199 INFO  (zkCallback-60-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 792199 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 792199 INFO  (zkCallback-58-thread-1-processing-n:127.0.0.1:34863_) [n:127.0.0.1:34863_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 792201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 792201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94656633793609737-127.0.0.1:53612_-n_0000000001
   [junit4]   2> 792202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94656633793609732-127.0.0.1:34863_-n_0000000000 to know if I could be the leader
   [junit4]   2> 792225 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 792225 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 792226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores
   [junit4]   2> 792227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, config=solrconfig.xml, collection=collection1, dataDir=data/, transient=false, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores/collection1/, name=collection1, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores/collection1, coreNodeName=, schema=schema.xml}
   [junit4]   2> 792227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores/collection1/
   [junit4]   2> 792227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 792228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 792228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A2ACEC8B599CE168]) [n:127.0.0.1:53612_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 792229 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 792229 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 792231 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 792231 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 792231 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 792232 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53612",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:53612_",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2"} current state version: 3
   [junit4]   2> 792232 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 792232 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53612",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:53612_",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 792232 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 792232 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 792336 INFO  (OverseerStateUpdate-94656633793609732-127.0.0.1:34863_-n_0000000000) [n:127.0.0.1:34863_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 792337 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 792337 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 793232 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 793232 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 793233 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 793233 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 793233 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 793234 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A2ACEC8B599CE168-001/shard-1-001/cores/collection1/'
   [junit4]   2> 793253 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 793260 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 793277 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 793325 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 793326 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 793336 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 793639 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 793650 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 793652 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 793668 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 793672 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 793675 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 793676 WARN  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 793676 WARN  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 793677 INFO  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 793677 WARN  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 793678 WARN  (coreLoadExecutor-375-thread-1-processing-n:127.0.0.1:53612_) [n:127.0.0.1:53612_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 793679 INFO  (coreLoadExecutor-375-thread-1-processing

[...truncated too long message...]

mit{dir=MockDirectoryWrapper(RAMDirectory@440450bc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@77508986),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@440450bc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@77508986),segFN=segments_3,generation=3}
   [junit4]   2> 4033060 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 4033061 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 4033110 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 4033111 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_A2ACEC8B599CE168-001/tempDir-001/shard6/cores/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_A2ACEC8B599CE168-001/tempDir-001/shard6/cores/collection1/data;done=false>>]
   [junit4]   2> 4033111 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_A2ACEC8B599CE168-001/tempDir-001/shard6/cores/collection1/data
   [junit4]   2> 4033111 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_A2ACEC8B599CE168-001/tempDir-001/shard6/cores/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_A2ACEC8B599CE168-001/tempDir-001/shard6/cores/collection1/data/index;done=false>>]
   [junit4]   2> 4033111 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_A2ACEC8B599CE168-001/tempDir-001/shard6/cores/collection1/data/index
   [junit4]   2> 4033113 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[A2ACEC8B599CE168]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@22b558d2{/,null,UNAVAILABLE}
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDistributedStatsComponentCardinality -Dtests.method=test -Dtests.seed=A2ACEC8B599CE168 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=th -Dtests.timezone=Asia/Riyadh -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    426s J2 | TestDistributedStatsComponentCardinality.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=114471, name=Thread-100086, state=RUNNABLE, group=TGRP-TestDistributedStatsComponentCardinality]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168:2AF8D351F7608C90]:0)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35068//collection1: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:57228//collection1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A2ACEC8B599CE168]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:624)
   [junit4]   2> 4033158 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[A2ACEC8B599CE168]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=th, timezone=Asia/Riyadh
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=251643320,total=530055168
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldResource, TestSmileRequest, StatelessScriptUpdateProcessorFactoryTest, TestNoOpRegenerator, TestPHPSerializedResponseWriter, TestJsonRequest, DateRangeFieldTest, TestReplicaProperties, SolrTestCaseJ4Test, SmileWriterTest, UnloadDistributedZkTest, TestReplicationHandlerBackup, DistributedExpandComponentTest, TestExceedMaxTermLength, TestCopyFieldCollectionResource, DistributedFacetPivotLongTailTest, QueryResultKeyTest, TestLeaderInitiatedRecoveryThread, TestSolrConfigHandlerCloud, TestSearchPerf, TestXIncludeConfig, DirectSolrConnectionTest, TermVectorComponentDistributedTest, TestSystemIdResolver, TestSchemaManager, TestSolrQueryParserResource, TestHashQParserPlugin, HdfsLockFactoryTest, HighlighterConfigTest, HdfsDirectoryFactoryTest, TestZkChroot, TestSolrDynamicMBean, TestConfigSetImmutable, DistribDocExpirationUpdateProcessorTest, TestCollapseQParserPlugin, ShardSplitTest, TestSortingResponseWriter, TestJmxIntegration, FieldAnalysisRequestHandlerTest, ClusterStateTest, HardAutoCommitTest, SolrCoreCheckLockOnStartupTest, HdfsBasicDistributedZkTest, TestCharFilters, CurrencyFieldXmlFileTest, UUIDFieldTest, TestJsonFacets, AutoCommitTest, TestElisionMultitermQuery, OverseerTest, RAMDirectoryFactoryTest, TestDistributedMissingSort, TestExtendedDismaxParser, PrimitiveFieldTypeTest, MergeStrategyTest, TestReRankQParserPlugin, SpatialHeatmapFacetsTest, TestFieldTypeResource, QueryParsingTest, CursorMarkTest, BufferStoreTest, BasicDistributedZk2Test, TestWriterPerf, SystemInfoHandlerTest, DocExpirationUpdateProcessorFactoryTest, TestIndexSearcher, SimpleFacetsTest, TestSolrIndexConfig, LeaderInitiatedRecoveryOnShardRestartTest, FieldMutatingUpdateProcessorTest, SpatialRPTFieldTypeTest, TestCSVResponseWriter, TestIndexingPerformance, TestStressRecovery, TestGroupingSearch, RulesTest, SuggesterTSTTest, ZkNodePropsTest, TestShardHandlerFactory, LeaderElectionTest, LukeRequestHandlerTest, BinaryUpdateRequestHandlerTest, SuggesterWFSTTest, CoreAdminCreateDiscoverTest, TestFastWriter, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, ClusterStateUpdateTest, TestRandomDVFaceting, ShardRoutingCustomTest, TestDistributedSearch, DistribJoinFromCollectionTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerRolesTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCollectionAPI, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestMiniSolrCloudClusterSSL, TestRandomRequestDistribution, TestShortCircuitedRequests, TestSolrCloudWithKerberosAlt, HdfsChaosMonkeySafeLeaderTest, HdfsNNFailoverTest, HdfsSyncSliceTest, HdfsWriteToMultipleCollectionsTest, TestReqParamsAPI, TestSolrConfigHandlerConcurrent, DistributedQueryComponentCustomSortTest, FacetPivotSmallTest, TestDistributedStatsComponentCardinality]
   [junit4] Completed [512/540] on J2 in 427.45s, 1 test, 1 error <<< FAILURES!

[...truncated 104 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 540 suites (7 ignored), 2131 tests, 4 errors, 29 ignored (17 assumptions) [seed: A2ACEC8B599CE168]

Total time: 191 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 979 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/979/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:44922: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44922: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=1543, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1543, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([C864271ED32EEAD5:403018C47DD2872D]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51661/s/sz: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([C864271ED32EEAD5]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=65282, name=coreZkRegister-5718-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=65282, name=coreZkRegister-5718-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([C864271ED32EEAD5]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10248 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/init-core-data-001
   [junit4]   2> 209943 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 209943 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /s/sz
   [junit4]   2> 210888 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 211975 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 212203 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 212213 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 212321 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 212389 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_36571_hdfs____31l16e/webapp
   [junit4]   2> 212653 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 213771 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36571
   [junit4]   2> 215546 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 215558 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 215573 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34626_datanode____.aaeilc/webapp
   [junit4]   2> 215732 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 216147 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34626
   [junit4]   2> 216280 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 216282 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 216298 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53239_datanode____.8p5rvj/webapp
   [junit4]   2> 216453 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 216807 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C864271ED32EEAD5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53239
   [junit4]   2> 217608 INFO  (IPC Server handler 5 on 37852) [    ] BlockStateChange BLOCK* processReport: from storage DS-4c67de93-63f1-4890-aa6c-90cbe672bb6d node DatanodeRegistration(127.0.0.1, datanodeUuid=ff1b8f5b-a13c-44b1-bebd-aee3fdd11264, infoPort=34626, ipcPort=34244, storageInfo=lv=-56;cid=testClusterID;nsid=1750495895;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 217608 INFO  (IPC Server handler 4 on 37852) [    ] BlockStateChange BLOCK* processReport: from storage DS-c440e419-3972-4774-a17d-06f7d8e44bed node DatanodeRegistration(127.0.0.1, datanodeUuid=3cac8066-e093-4ab8-a7be-c448c50329a2, infoPort=53239, ipcPort=37262, storageInfo=lv=-56;cid=testClusterID;nsid=1750495895;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 217609 INFO  (IPC Server handler 5 on 37852) [    ] BlockStateChange BLOCK* processReport: from storage DS-36e86dec-4af3-466e-8092-0bb97d8b2f01 node DatanodeRegistration(127.0.0.1, datanodeUuid=ff1b8f5b-a13c-44b1-bebd-aee3fdd11264, infoPort=34626, ipcPort=34244, storageInfo=lv=-56;cid=testClusterID;nsid=1750495895;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 217609 INFO  (IPC Server handler 4 on 37852) [    ] BlockStateChange BLOCK* processReport: from storage DS-b829ee1f-e174-4e32-b241-241aca8f2a97 node DatanodeRegistration(127.0.0.1, datanodeUuid=3cac8066-e093-4ab8-a7be-c448c50329a2, infoPort=53239, ipcPort=37262, storageInfo=lv=-56;cid=testClusterID;nsid=1750495895;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 218217 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 218217 INFO  (Thread-220) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 218217 INFO  (Thread-220) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 218317 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.ZkTestServer start zk server on port:34530
   [junit4]   2> 218317 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 218318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 218321 INFO  (zkCallback-38-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43749ce2 name:ZooKeeperConnection Watcher:127.0.0.1:34530 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 218321 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 218322 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 218322 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 218326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 218327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 218328 INFO  (zkCallback-39-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5fa2d0d3 name:ZooKeeperConnection Watcher:127.0.0.1:34530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 218328 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 218328 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 218328 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 218332 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 218334 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 218335 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 218337 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 218337 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 218340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 218341 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 218343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 218343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 218345 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 218346 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 218348 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 218348 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 218350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 218350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 218352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 218352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 218354 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 218354 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 218356 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 218356 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 218358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 218358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 218360 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 218360 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 218368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 218369 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 218370 INFO  (zkCallback-40-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72ca8ad4 name:ZooKeeperConnection Watcher:127.0.0.1:34530 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 218370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 218371 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 218371 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 218376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 218376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 218378 INFO  (zkCallback-41-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3645452f name:ZooKeeperConnection Watcher:127.0.0.1:34530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 218378 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 218378 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 218379 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 218379 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 218383 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 218383 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 218385 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 218385 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 218387 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 218387 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 218389 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 218390 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 218391 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 218392 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 218394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 218394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 218396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 218396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 218398 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 218399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 218401 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 218401 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 218403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 218403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 218405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 218405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 218743 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores/collection1
   [junit4]   2> 218745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 218749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a2d484d{/s/sz,null,AVAILABLE}
   [junit4]   2> 218749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@52e5e57b{HTTP/1.1}{127.0.0.1:51661}
   [junit4]   2> 218750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.Server Started @221803ms
   [junit4]   2> 218750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/s/sz, solr.data.dir=hdfs://localhost:37852/hdfs__localhost_37852__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001_tempDir-002_control_data, hostPort=51661, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores}
   [junit4]   2> 218750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 218750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/'
   [junit4]   2> 218770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 218770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 218773 INFO  (zkCallback-42-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75a4c4b2 name:ZooKeeperConnection Watcher:127.0.0.1:34530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 218773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 218774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 218775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 218775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/solr.xml
   [junit4]   2> 218793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores
   [junit4]   2> 218793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.CoreContainer New CoreContainer 417400814
   [junit4]   2> 218793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/]
   [junit4]   2> 218793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/lib
   [junit4]   2> 218793 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/lib).
   [junit4]   2> 218802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 218803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 218805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 218805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 218805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34530/solr
   [junit4]   2> 218805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 218805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 218805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 218807 INFO  (zkCallback-44-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58e3e3f7 name:ZooKeeperConnection Watcher:127.0.0.1:34530 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 218808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 218808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 218816 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 218817 INFO  (zkCallback-45-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57bb4a08 name:ZooKeeperConnection Watcher:127.0.0.1:34530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 218818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 218820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 218822 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 218826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 218829 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 218831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 218834 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 218835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 218837 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 218839 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 218840 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51661_s%2Fsz
   [junit4]   2> 218841 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51661_s%2Fsz
   [junit4]   2> 218842 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 218844 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 218846 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 218846 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000
   [junit4]   2> 218847 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51661_s%2Fsz
   [junit4]   2> 218847 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 218848 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer Overseer (id=94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) starting
   [junit4]   2> 218850 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 218857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 218858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 218858 INFO  (OverseerCollectionConfigSetProcessor-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 218858 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 218889 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 218890 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 218891 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores
   [junit4]   2> 218892 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores/collection1, configSetProperties=configsetprops.json, transient=false, coreNodeName=, loadOnStartup=true, collection=control_collection, shard=, config=solrconfig.xml, name=collection1, dataDir=data/, schema=schema.xml}
   [junit4]   2> 218892 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores/collection1/
   [junit4]   2> 218892 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 218900 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 218901 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 218901 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 218901 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 218903 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 218903 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51661/s/sz",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:51661_s%2Fsz",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 218904 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51661/s/sz",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:51661_s%2Fsz",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 218904 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 218905 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 218906 INFO  (zkCallback-45-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 219903 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 219903 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 219904 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 219904 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 219905 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 219905 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores/collection1/'
   [junit4]   2> 219922 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 219934 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 219950 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 220000 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 220001 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 220025 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 220334 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 220350 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 220353 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 220386 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 220391 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 220397 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 220398 WARN  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 220398 WARN  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 220398 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 220399 WARN  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 220399 WARN  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 220400 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 220400 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 220401 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 220416 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37852/solr_hdfs_home
   [junit4]   2> 220416 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 220416 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 220416 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6adfd28b
   [junit4]   2> 220417 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37852/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 220540 WARN  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 225589 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37852/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 225596 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:37852/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 225634 WARN  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:37852/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 225634 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37852/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 225657 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37852/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 225658 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6e0f71a
   [junit4]   2> 226061 INFO  (IPC Server handler 6 on 37852) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36100 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-c440e419-3972-4774-a17d-06f7d8e44bed:NORMAL:127.0.0.1:36100|RBW], ReplicaUnderConstruction[[DISK]DS-36e86dec-4af3-466e-8092-0bb97d8b2f01:NORMAL:127.0.0.1:46225|RBW]]} size 71
   [junit4]   2> 226062 INFO  (IPC Server handler 7 on 37852) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46225 is added to blk_1073741825_1001 size 71
   [junit4]   2> 226471 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b68f0262 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a2c76a4; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 226471 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 226475 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:37852/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 226480 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 226481 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 226481 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 226481 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 226481 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 226481 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 226482 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 226482 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 226482 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 226485 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 226486 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 226488 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 226489 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 226495 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/config,/update/json/docs,/update/csv,/admin/mbeans,/schema,/update/json,standard,/admin/system,/admin/segments,/admin/luke,/update,/admin/file,/admin/logging,/get,/admin/threads,/admin/plugins,/admin/ping,/admin/properties
   [junit4]   2> 226499 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 226499 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 226499 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 226518 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 226518 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 226520 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@59a4b377
   [junit4]   2> 226615 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b68f0262 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a2c76a4; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 226615 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 226615 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4a6e22c4[collection1] main
   [junit4]   2> 226616 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 226617 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 226617 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 226618 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 226618 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 226618 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 226619 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 226619 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 226619 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 226623 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 226624 INFO  (searcherExecutor-126-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4a6e22c4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 226626 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 226626 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 226626 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 226626 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514441210634174464
   [junit4]   2> 226626 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1514441210634174464
   [junit4]   2> 226626 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 226627 INFO  (coreLoadExecutor-125-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 226628 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:51661/s/sz collection:control_collection shard:shard1
   [junit4]   2> 226629 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 226634 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94652575113740294-core_node1-n_0000000000
   [junit4]   2> 226634 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 226637 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 226638 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 226638 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 226638 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51661/s/sz/collection1/
   [junit4]   2> 226638 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 226638 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51661/s/sz/collection1/ has no replicas
   [junit4]   2> 226638 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51661/s/sz/collection1/ shard1
   [junit4]   2> 226638 INFO  (zkCallback-45-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 226639 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 226642 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 226646 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51661/s/sz",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 226749 INFO  (zkCallback-45-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 226798 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:51661/s/sz/collection1/ and leader is http://127.0.0.1:51661/s/sz/collection1/
   [junit4]   2> 226798 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51661/s/sz
   [junit4]   2> 226798 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 226798 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 226798 INFO  (coreZkRegister-119-thread-1-processing-s:shard1 x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 226800 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51661/s/sz",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:51661_s%2Fsz",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 226801 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51661/s/sz",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:51661_s%2Fsz",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active"}
   [junit4]   2> 226904 INFO  (zkCallback-45-thread-1-processing-n:127.0.0.1:51661_s%2Fsz) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 226905 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 226906 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 226907 INFO  (zkCallback-47-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@232fb5d5 name:ZooKeeperConnection Watcher:127.0.0.1:34530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 226907 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 226909 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 226909 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 226912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 226912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 226912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 226913 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 226914 INFO  (zkCallback-48-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@d21e7ed name:ZooKeeperConnection Watcher:127.0.0.1:34530/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 226914 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 226915 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 226918 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x1504603fcbb0008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 226920 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 226921 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 227024 INFO  (OverseerStateUpdate-94652575113740294-127.0.0.1:51661_s%2Fsz-n_0000000000) [n:127.0.0.1:51661_s%2Fsz    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 227230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/shard-1-001/cores/collection1
   [junit4]   2> 227230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/shard-1-001
   [junit4]   2> 227232 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 227234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@51a8a526{/s/sz,null,AVAILABLE}
   [junit4]   2> 227234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@16f28bac{HTTP/1.1}{127.0.0.1:53111}
   [junit4]   2> 227234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.e.j.s.Server Started @230287ms
   [junit4]   2> 227234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/shard-1-001/cores, hostPort=53111, solr.data.dir=hdfs://localhost:37852/hdfs__localhost_37852__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/s/sz}
   [junit4]   2> 227235 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 227235 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C864271ED32EEAD5-001/shard-1-001/'
   [junit4]   2> 227252 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C864271ED32EEAD5]) [    ] o.a.s.c.c.SolrZkClient Usin

[...truncated too long message...]

xtHandler Stopped o.e.j.s.ServletContextHandler@7b6f83ad{/_zg,null,UNAVAILABLE}
   [junit4]   2> 3954591 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[C864271ED32EEAD5]) [n:127.0.0.1:34080__zg c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37808 37808
   [junit4]   2> 3954760 INFO  (Thread-57882) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37808 37808
   [junit4]   2> 3954761 WARN  (Thread-57882) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	20	/solr/collections/collection1/state.json
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94652796130820100-127.0.0.1:44323__zg-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=C864271ED32EEAD5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_PE -Dtests.timezone=Europe/Warsaw -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    364s J0 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=65282, name=coreZkRegister-5718-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C864271ED32EEAD5]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3954778 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[C864271ED32EEAD5]-worker) [n:127.0.0.1:34080__zg c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_C864271ED32EEAD5-001
   [junit4]   2> Oct 08, 2015 2:56:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=es_PE, timezone=Europe/Warsaw
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=116100200,total=515375104
   [junit4]   2> NOTE: All tests run in this JVM: [TestBulkSchemaConcurrent, TestMacros, LeaderElectionIntegrationTest, TestFreeTextSuggestions, TestManagedStopFilterFactory, CacheHeaderTest, TestFieldSortValues, PrimUtilsTest, AnalysisAfterCoreReloadTest, ExternalFileFieldSortTest, TestValueSourceCache, ReturnFieldsTest, TestManagedSchema, DistributedIntervalFacetingTest, DistributedDebugComponentTest, CoreMergeIndexesAdminHandlerTest, BlockCacheTest, TestSolr4Spatial, TestComplexPhraseQParserPlugin, JSONWriterTest, TestExclusionRuleCollectionAccess, CustomCollectionTest, PeerSyncTest, CollectionReloadTest, RemoteQueryErrorTest, DeleteInactiveReplicaTest, TimeZoneUtilsTest, TestChildDocTransformer, StandardRequestHandlerTest, LeaderInitiatedRecoveryOnCommitTest, TestSchemaSimilarityResource, TestAtomicUpdateErrorCases, TestClusterStateMutator, DistributedSpellCheckComponentTest, TestConfigReload, TestRangeQuery, CollectionsAPIAsyncDistributedZkTest, TestDistribDocBasedVersion, DistributedMLTComponentTest, OutputWriterTest, TestRealTimeGet, SharedFSAutoReplicaFailoverUtilsTest, QueryEqualityTest, DateMathParserTest, DefaultValueUpdateProcessorTest, TestJoin, TestSolrJ, TestRebalanceLeaders, SynonymTokenizerTest, OverseerStatusTest, TestOrdValues, DistributedFacetPivotLargeTest, SyncSliceTest, TestStressUserVersions, TestLuceneMatchVersion, DataDrivenBlockJoinTest, TestAuthenticationFramework, HdfsDirectoryTest, DisMaxRequestHandlerTest, TestConfigSetProperties, TestComponentsName, TestManagedSchemaDynamicFieldResource, BlockDirectoryTest, TestCollationFieldDocValues, TestInfoStreamLogging, SpellingQueryConverterTest, TriLevelCompositeIdRoutingTest, TestRemoteStreaming, TestLRUStatsCache, PolyFieldTest, AliasIntegrationTest, AlternateDirectoryTest, TestMiniSolrCloudClusterSSL, TestReplicationHandler, TestFunctionQuery, BasicDistributedZkTest, OpenExchangeRatesOrgProviderTest, ExplicitHLLTest, TestRandomRequestDistribution, TestTrieFacet, TestPseudoReturnFields, TestCollectionAPI, TestExpandComponent, CursorPagingTest, TestMaxScoreQueryParser, TestTrackingShardHandlerFactory, DistributedQueryElevationComponentTest, ParsingFieldUpdateProcessorsTest, OpenCloseCoreStressTest, BasicZkTest, RecoveryZkTest, TestRandomFaceting, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, TestConfig, TestSolrCoreProperties, TestPhraseSuggestions, TestCodecSupport, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TestFastOutputStream, TestFastLRUCache, TestSuggestSpellingConverter, TestUtils, TestDocumentBuilder, SliceStateTest, UUIDFieldTest, CircularListTest, TestRTGBase, TestCrossCoreJoin, AssignTest, AsyncCallRequestStatusResponseTest, CollectionStateFormat2Test, ConcurrentDeleteAndCreateCollectionTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [437/540] on J0 in 364.72s, 1 test, 1 error <<< FAILURES!

[...truncated 334 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 540 suites (7 ignored), 2131 tests, 3 errors, 29 ignored (17 assumptions) [seed: C864271ED32EEAD5]

Total time: 188 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 978 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/978/

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=43659, name=coreZkRegister-565-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43659, name=coreZkRegister-565-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([730B792D2F31E8E]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10272 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/init-core-data-001
   [junit4]   2> 550628 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[730B792D2F31E8E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /dss/s
   [junit4]   2> 550638 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 550655 INFO  (Thread-42667) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 550655 INFO  (Thread-42667) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 550755 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.ZkTestServer start zk server on port:54960
   [junit4]   2> 550756 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 550757 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 550760 INFO  (zkCallback-149-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@142807fc name:ZooKeeperConnection Watcher:127.0.0.1:54960 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 550760 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 550761 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 550761 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 550766 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 550767 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 550769 INFO  (zkCallback-150-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ce73cca name:ZooKeeperConnection Watcher:127.0.0.1:54960/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 550769 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 550769 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 550769 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 550773 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 550775 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 550778 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 550780 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 550780 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 550784 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 550784 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 550786 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 550787 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 550790 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 550790 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 550792 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 550792 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 550794 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 550794 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 550796 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 550797 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 550799 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 550799 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 550801 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 550801 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 550804 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 550804 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 550806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 550806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 551197 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1
   [junit4]   2> 551200 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 551230 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d3f209e{/dss/s,null,AVAILABLE}
   [junit4]   2> 551232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@58443caa{HTTP/1.1}{127.0.0.1:35257}
   [junit4]   2> 551232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.Server Started @554707ms
   [junit4]   2> 551232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/dss/s, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/tempDir-001/control/data, hostPort=35257}
   [junit4]   2> 551232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 551232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/'
   [junit4]   2> 551253 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 551254 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 551258 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40fab7c6 name:ZooKeeperConnection Watcher:127.0.0.1:54960/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 551258 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 551258 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 551263 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 551264 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/solr.xml
   [junit4]   2> 551292 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores
   [junit4]   2> 551292 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoreContainer New CoreContainer 519392268
   [junit4]   2> 551292 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/]
   [junit4]   2> 551293 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/lib
   [junit4]   2> 551293 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/lib).
   [junit4]   2> 551304 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 551306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 551306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 551306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 551306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54960/solr
   [junit4]   2> 551306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 551307 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 551308 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 551311 INFO  (zkCallback-153-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b270210 name:ZooKeeperConnection Watcher:127.0.0.1:54960 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 551312 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 551312 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 551315 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 551318 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48954319 name:ZooKeeperConnection Watcher:127.0.0.1:54960/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 551318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 551321 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 551325 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 551331 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 551334 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 551338 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 551342 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 551344 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 551349 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 551356 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 551359 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35257_dss%2Fs
   [junit4]   2> 551360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35257_dss%2Fs
   [junit4]   2> 551365 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 551370 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 551372 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 551380 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000
   [junit4]   2> 551381 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35257_dss%2Fs
   [junit4]   2> 551381 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 551393 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer Overseer (id=94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) starting
   [junit4]   2> 551397 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 551414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 551415 INFO  (OverseerCollectionConfigSetProcessor-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 551415 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 551416 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 551439 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 551439 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 551440 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores
   [junit4]   2> 551441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, configSetProperties=configsetprops.json, shard=, dataDir=data/, collection=control_collection, transient=false, config=solrconfig.xml, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/, name=collection1}
   [junit4]   2> 551441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/
   [junit4]   2> 551441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 551443 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 551443 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 551443 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 551444 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 551446 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35257/dss/s",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:35257_dss%2Fs",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 551447 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 551447 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35257/dss/s",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:35257_dss%2Fs",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 551448 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 551448 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 551451 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 552448 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 552448 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 552450 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 552450 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 552451 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 552451 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/'
   [junit4]   2> 552474 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 552488 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 552506 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 552543 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 552544 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 552553 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 553022 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 553038 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 553040 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 553060 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 553064 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 553069 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 553070 WARN  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 553071 WARN  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 553071 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 553072 WARN  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 553072 WARN  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 553073 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 553073 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 553073 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 553073 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 553074 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6393a5a4
   [junit4]   2> 553074 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/data
   [junit4]   2> 553074 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/data/index/
   [junit4]   2> 553074 WARN  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 553075 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/data/index
   [junit4]   2> 553075 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=45, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 553080 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@765e0b80 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@18453d36; maxCacheMB=0.39453125 maxMergeSizeMB=0.3037109375)),segFN=segments_1,generation=1}
   [junit4]   2> 553080 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 553083 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/data/
   [junit4]   2> 553083 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 553094 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 553094 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 553094 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 553094 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 553094 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 553095 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 553095 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 553095 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 553095 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 553103 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 553104 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 553106 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 553108 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 553112 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/properties,/get,/admin/luke,/admin/logging,/admin/system,/update/csv,/config,/schema,/replication,/admin/segments,/update/json,/update/json/docs,/update,/admin/threads,/admin/ping,/admin/file,/admin/mbeans,/admin/plugins
   [junit4]   2> 553114 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 553115 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 553115 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 553116 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 553116 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 553116 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=92.5, floorSegmentMB=0.6611328125, forceMergeDeletesPctAllowed=25.341203402105496, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.715005091231669
   [junit4]   2> 553117 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@765e0b80 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@18453d36; maxCacheMB=0.39453125 maxMergeSizeMB=0.3037109375)),segFN=segments_1,generation=1}
   [junit4]   2> 553117 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 553117 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@779231ee[collection1] main
   [junit4]   2> 553117 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 553118 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 553118 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 553119 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 553119 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 553119 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 553119 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 553119 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 553119 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 553120 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 553123 INFO  (searcherExecutor-528-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@779231ee[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 553123 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 553123 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 553123 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 553123 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514324801591705600
   [junit4]   2> 553125 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514324801591705600
   [junit4]   2> 553126 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 553128 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 553129 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:35257/dss/s collection:control_collection shard:shard1
   [junit4]   2> 553131 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 553138 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94645299939639300-core_node1-n_0000000000
   [junit4]   2> 553139 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 553141 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 553141 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 553141 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35257/dss/s/collection1/
   [junit4]   2> 553141 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 553141 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35257/dss/s/collection1/ has no replicas
   [junit4]   2> 553141 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35257/dss/s/collection1/ shard1
   [junit4]   2> 553142 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 553142 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 553143 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 553147 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 553150 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35257/dss/s",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 553255 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 553302 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:35257/dss/s/collection1/ and leader is http://127.0.0.1:35257/dss/s/collection1/
   [junit4]   2> 553302 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35257/dss/s
   [junit4]   2> 553303 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 553303 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 553303 INFO  (coreZkRegister-521-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:35257_dss%2Fs x:collection1) [n:127.0.0.1:35257_dss%2Fs c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 553308 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35257/dss/s",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:35257_dss%2Fs",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 553309 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35257/dss/s",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:35257_dss%2Fs",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 553416 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 553448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 553451 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 553453 INFO  (zkCallback-156-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24bcb9b1 name:ZooKeeperConnection Watcher:127.0.0.1:54960/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 553453 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 553453 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 553453 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 553461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 553825 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores/collection1
   [junit4]   2> 553826 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001
   [junit4]   2> 553827 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 553832 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7bba7009{/dss/s,null,AVAILABLE}
   [junit4]   2> 553833 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7fb1e526{HTTP/1.1}{127.0.0.1:49132}
   [junit4]   2> 553833 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.Server Started @557308ms
   [junit4]   2> 553833 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/dss/s, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/tempDir-001/jetty1, hostPort=49132, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 553833 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 553833 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/'
   [junit4]   2> 553869 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 553870 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 553872 INFO  (zkCallback-157-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@16963a99 name:ZooKeeperConnection Watcher:127.0.0.1:54960/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 553872 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 553872 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 553877 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 553877 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/solr.xml
   [junit4]   2> 553894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores
   [junit4]   2> 553894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoreContainer New CoreContainer 547398303
   [junit4]   2> 553894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/]
   [junit4]   2> 553895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/lib
   [junit4]   2> 553895 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/lib).
   [junit4]   2> 553903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 553905 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 553906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 553906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 553906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54960/solr
   [junit4]   2> 553906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 553906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 553909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 553910 INFO  (zkCallback-159-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7ff23e9b name:ZooKeeperConnection Watcher:127.0.0.1:54960 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 553911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 553912 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 553915 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 553915 INFO  (zkCallback-160-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4f3e3ad1 name:ZooKeeperConnection Watcher:127.0.0.1:54960/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 553916 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 553925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 554936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49132_dss%2Fs
   [junit4]   2> 554937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49132_dss%2Fs
   [junit4]   2> 554938 INFO  (zkCallback-156-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 554938 INFO  (zkCallback-160-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 554938 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 554941 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 554941 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94645299939639304-127.0.0.1:49132_dss%2Fs-n_0000000001
   [junit4]   2> 554942 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000 to know if I could be the leader
   [junit4]   2> 554970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 554970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 554971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores
   [junit4]   2> 554972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores/collection1/, coreNodeName=, loadOnStartup=true, config=solrconfig.xml, name=collection1, dataDir=data/, collection=collection1, transient=false, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores/collection1}
   [junit4]   2> 554972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores/collection1/
   [junit4]   2> 554972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 554974 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 554974 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 554974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 554974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 554976 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49132/dss/s",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:49132_dss%2Fs",
   [junit4]   2>   "core":"collection1"} current state version: 4
   [junit4]   2> 554977 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49132/dss/s",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:49132_dss%2Fs",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 554977 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 554977 INFO  (OverseerStateUpdate-94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 554978 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 554978 INFO  (zkCallback-156-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 554979 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:35257_dss%2Fs) [n:127.0.0.1:35257_dss%2Fs    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 554983 INFO  (zkCallback-160-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 555980 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 555980 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 555981 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 555981 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 555982 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 555982 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores/collection1/'
   [junit4]   2> 556004 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 556011 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 556025 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 556241 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 556263 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 556274 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 557250 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 557260 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 557262 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 557281 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 557284 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 557288 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 557288 WARN  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 557288 WARN  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 557289 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 557289 WARN  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 557289 WARN  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 557290 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 557290 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 557291 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 557291 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 557291 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6393a5a4
   [junit4]   2> 557292 INFO  (coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:49132_dss%2Fs) [n:127.0.0.1:49132_dss%2Fs c:collection1 s:shard1 r:core_node1

[...truncated too long message...]

rd1/election/94645299939639314-core_node1-n_0000000004
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:166)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:268)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:450)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 926435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@5deca3a8{HTTP/1.1}{127.0.0.1:34107}
   [junit4]   2> 926436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@ee8db0{/dss/s,null,UNAVAILABLE}
   [junit4]   2> 926437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[730B792D2F31E8E]) [n:127.0.0.1:34107_dss%2Fs c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54960 54960
   [junit4]   2> 926615 INFO  (Thread-42667) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54960 54960
   [junit4]   2> 926616 WARN  (Thread-42667) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94645299939639300-127.0.0.1:35257_dss%2Fs-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=730B792D2F31E8E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_VE -Dtests.timezone=US/Michigan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR    376s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43659, name=coreZkRegister-565-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([730B792D2F31E8E]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 926648 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[730B792D2F31E8E]-worker) [n:127.0.0.1:34107_dss%2Fs c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_730B792D2F31E8E-001
   [junit4]   2> Oct 06, 2015 7:10:43 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=es_VE, timezone=US/Michigan
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=240392800,total=419954688
   [junit4]   2> NOTE: All tests run in this JVM: [TestSchemaManager, HighlighterConfigTest, TestCollapseQParserPlugin, BasicAuthIntegrationTest, HdfsBasicDistributedZkTest, CurrencyFieldXmlFileTest, ActionThrottleTest, TestLeaderElectionZkExpiry, TestReversedWildcardFilterFactory, BasicDistributedZk2Test, TestSort, DirectUpdateHandlerOptimizeTest, TestNoOpRegenerator, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [105/540] on J2 in 376.58s, 1 test, 1 error <<< FAILURES!

[...truncated 1408 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 540 suites (7 ignored), 2130 tests, 1 error, 29 ignored (17 assumptions) [seed: 730B792D2F31E8E]

Total time: 182 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 977 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/977/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=18289, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=18289, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39969: collection already exists: awholynewstresscollection_collection3_1
	at __randomizedtesting.SeedInfo.seed([F1C31F1CCECA8AE8]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=7640, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=7640, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([F1C31F1CCECA8AE8:799720C66036E710]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42023: Could not find collection : awholynewstresscollection_collection1_4
	at __randomizedtesting.SeedInfo.seed([F1C31F1CCECA8AE8]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=116935, name=coreZkRegister-6107-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=116935, name=coreZkRegister-6107-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([F1C31F1CCECA8AE8]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:422)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10470 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/init-core-data-001
   [junit4]   2> 682384 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 682384 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 682434 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 682444 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 682448 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 682466 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_42568_hdfs____.38yeal/webapp
   [junit4]   2> 682593 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 683150 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42568
   [junit4]   2> 683241 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 683244 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 683256 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_48541_datanode____vygzmd/webapp
   [junit4]   2> 683385 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 683610 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48541
   [junit4]   2> 683673 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 683679 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 684447 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=272715603
   [junit4]   2> 684460 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45058_datanode____wczxjt/webapp
   [junit4]   2> 684499 INFO  (IPC Server handler 9 on 56115) [    ] BlockStateChange BLOCK* processReport: from storage DS-25d7982f-ab64-421c-bb84-cde50e29ff07 node DatanodeRegistration(127.0.0.1, datanodeUuid=0840c058-017f-43f1-b1ec-6316166ca01b, infoPort=48541, ipcPort=34104, storageInfo=lv=-56;cid=testClusterID;nsid=1369767611;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 684499 INFO  (IPC Server handler 9 on 56115) [    ] BlockStateChange BLOCK* processReport: from storage DS-bfd42281-035a-485e-87a0-168455cbda86 node DatanodeRegistration(127.0.0.1, datanodeUuid=0840c058-017f-43f1-b1ec-6316166ca01b, infoPort=48541, ipcPort=34104, storageInfo=lv=-56;cid=testClusterID;nsid=1369767611;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 684606 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 684798 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1C31F1CCECA8AE8]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45058
   [junit4]   2> 684875 INFO  (IPC Server handler 3 on 56115) [    ] BlockStateChange BLOCK* processReport: from storage DS-619c130b-7b64-4cce-acd8-438876fec21c node DatanodeRegistration(127.0.0.1, datanodeUuid=e6f7eac4-940a-4a97-b2a7-84a54dd4add3, infoPort=45058, ipcPort=46107, storageInfo=lv=-56;cid=testClusterID;nsid=1369767611;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 684875 INFO  (IPC Server handler 3 on 56115) [    ] BlockStateChange BLOCK* processReport: from storage DS-52a128ba-170a-4eb1-9d42-1c3c53292377 node DatanodeRegistration(127.0.0.1, datanodeUuid=e6f7eac4-940a-4a97-b2a7-84a54dd4add3, infoPort=45058, ipcPort=46107, storageInfo=lv=-56;cid=testClusterID;nsid=1369767611;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 684984 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 684985 INFO  (Thread-3964) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 684985 INFO  (Thread-3964) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 685085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.ZkTestServer start zk server on port:44237
   [junit4]   2> 685085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 685085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 685093 INFO  (zkCallback-358-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@269d4d64 name:ZooKeeperConnection Watcher:127.0.0.1:44237 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685094 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 685094 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 685094 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 685097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 685097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 685099 INFO  (zkCallback-359-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3ac931d9 name:ZooKeeperConnection Watcher:127.0.0.1:44237/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 685099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 685099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 685101 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 685103 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 685104 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 685106 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 685106 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 685110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 685111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 685114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 685114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 685116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 685116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 685117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 685118 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 685119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 685119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 685121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 685122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 685123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 685123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 685125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 685125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 685127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 685127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 685128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 685129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 685131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 685131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 685133 INFO  (zkCallback-360-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7236e0a6 name:ZooKeeperConnection Watcher:127.0.0.1:44237 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 685133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 685133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 685135 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 685135 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 685137 INFO  (zkCallback-361-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@905053e name:ZooKeeperConnection Watcher:127.0.0.1:44237/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 685137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 685137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 685137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 685140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 685140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 685142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 685142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 685144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 685144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 685146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 685147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 685152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 685153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 685154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 685155 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 685156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 685156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 685159 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 685159 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 685161 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 685161 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 685162 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 685163 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 685164 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 685164 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 685461 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores/collection1
   [junit4]   2> 685463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 685465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@364168ce{/,null,AVAILABLE}
   [junit4]   2> 685465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@47c836{HTTP/1.1}{127.0.0.1:49410}
   [junit4]   2> 685465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.Server Started @688711ms
   [junit4]   2> 685465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=hdfs://localhost:56115/hdfs__localhost_56115__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores, hostPort=49410}
   [junit4]   2> 685466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 685466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/'
   [junit4]   2> 685482 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 685483 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 685484 INFO  (zkCallback-362-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e7657f1 name:ZooKeeperConnection Watcher:127.0.0.1:44237/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685484 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 685484 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 685485 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 685486 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/solr.xml
   [junit4]   2> 685500 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores
   [junit4]   2> 685500 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1801156008
   [junit4]   2> 685500 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/]
   [junit4]   2> 685501 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/lib
   [junit4]   2> 685501 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/lib).
   [junit4]   2> 685509 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 685510 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 685510 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 685511 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 685511 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44237/solr
   [junit4]   2> 685511 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 685511 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 685511 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 685513 INFO  (zkCallback-364-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78825153 name:ZooKeeperConnection Watcher:127.0.0.1:44237 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 685514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 685515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 685517 INFO  (zkCallback-365-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7dcc5bc8 name:ZooKeeperConnection Watcher:127.0.0.1:44237/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685517 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 685518 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 685520 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 685523 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 685525 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 685527 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 685529 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 685530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 685532 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 685533 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 685534 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49410_
   [junit4]   2> 685535 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49410_
   [junit4]   2> 685536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 685538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 685539 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 685539 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94639553585283078-127.0.0.1:49410_-n_0000000000
   [junit4]   2> 685541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49410_
   [junit4]   2> 685541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 685542 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer Overseer (id=94639553585283078-127.0.0.1:49410_-n_0000000000) starting
   [junit4]   2> 685544 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 685551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 685552 INFO  (OverseerCollectionConfigSetProcessor-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 685552 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 685552 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 685577 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 685578 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 685579 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores
   [junit4]   2> 685579 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores/collection1/, transient=false, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, config=solrconfig.xml, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores/collection1, shard=, name=collection1, collection=control_collection, loadOnStartup=true}
   [junit4]   2> 685579 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores/collection1/
   [junit4]   2> 685580 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 685581 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 685581 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 685581 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 685581 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [n:127.0.0.1:49410_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 685584 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49410",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:49410_"} current state version: 0
   [junit4]   2> 685586 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 685586 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49410",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:49410_"}
   [junit4]   2> 685587 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 685587 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 685589 INFO  (zkCallback-365-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 686587 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 686587 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 686587 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 686587 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 686588 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 686588 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores/collection1/'
   [junit4]   2> 686605 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 686612 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 686627 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 686658 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 686659 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 686666 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 686953 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 686963 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 686965 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 686980 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 686984 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 686987 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 686988 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 686988 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 686988 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 686989 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 686989 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 686990 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 686990 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 686990 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 686991 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56115/solr_hdfs_home
   [junit4]   2> 686991 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 686991 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 686991 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e5bf9e7
   [junit4]   2> 686992 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56115/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 687009 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:56115/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 687011 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:56115/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 687025 WARN  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:56115/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 687025 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56115/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 687042 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:56115/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 687043 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@28e94f58
   [junit4]   2> 687072 INFO  (IPC Server handler 6 on 56115) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:48994 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-52a128ba-170a-4eb1-9d42-1c3c53292377:NORMAL:127.0.0.1:36235|RBW], ReplicaUnderConstruction[[DISK]DS-25d7982f-ab64-421c-bb84-cde50e29ff07:NORMAL:127.0.0.1:48994|RBW]]} size 0
   [junit4]   2> 687075 INFO  (IPC Server handler 5 on 56115) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36235 is added to blk_1073741825_1001 size 71
   [junit4]   2> 687078 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8ae23bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d8278bd; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 687078 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 687080 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:56115/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 687087 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 687088 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 687088 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 687088 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 687088 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 687088 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 687089 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 687089 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 687089 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 687092 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 687094 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 687097 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 687103 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 687109 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/admin/threads,/replication,/update/json/docs,/admin/system,/config,/admin/properties,/admin/luke,/admin/plugins,/admin/segments,/admin/file,/schema,/get,/update/json,standard,/admin/logging,/update,/admin/ping,/admin/mbeans
   [junit4]   2> 687111 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 687111 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 687112 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 687128 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 687128 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 687130 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=14.75390625, floorSegmentMB=1.2119140625, forceMergeDeletesPctAllowed=24.461639842302144, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 687140 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8ae23bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d8278bd; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 687140 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 687140 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5b2c49b9[collection1] main
   [junit4]   2> 687141 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 687142 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 687142 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 687142 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 687142 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 687142 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 687143 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 687143 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 687143 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 687143 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 687145 INFO  (searcherExecutor-1032-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5b2c49b9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 687145 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 687145 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 687145 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 687145 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514232859628404736
   [junit4]   2> 687145 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1514232859628404736
   [junit4]   2> 687145 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 687146 INFO  (coreLoadExecutor-1031-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 687146 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:49410 collection:control_collection shard:shard1
   [junit4]   2> 687147 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 687152 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94639553585283078-core_node1-n_0000000000
   [junit4]   2> 687152 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 687154 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 687155 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 687155 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49410/collection1/
   [junit4]   2> 687155 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 687155 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 687155 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:49410/collection1/ has no replicas
   [junit4]   2> 687155 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49410/collection1/ shard1
   [junit4]   2> 687156 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 687156 INFO  (zkCallback-365-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 687161 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 687164 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:49410",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 687267 INFO  (zkCallback-365-thread-1-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 687315 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:49410/collection1/ and leader is http://127.0.0.1:49410/collection1/
   [junit4]   2> 687316 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49410
   [junit4]   2> 687316 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 687316 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 687316 INFO  (coreZkRegister-1025-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:49410_) [n:127.0.0.1:49410_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 687319 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:49410",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:49410_"} current state version: 3
   [junit4]   2> 687319 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:49410",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:49410_"}
   [junit4]   2> 687422 INFO  (zkCallback-365-thread-2-processing-n:127.0.0.1:49410_) [n:127.0.0.1:49410_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 687584 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 687585 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 687588 INFO  (zkCallback-367-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c5ea897 name:ZooKeeperConnection Watcher:127.0.0.1:44237/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 687588 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 687589 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 687589 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 687592 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 687592 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 687592 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 687595 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 687596 INFO  (zkCallback-368-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49e3c6d name:ZooKeeperConnection Watcher:127.0.0.1:44237/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 687596 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 687596 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 687599 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 687599 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 687702 INFO  (OverseerStateUpdate-94639553585283078-127.0.0.1:49410_-n_0000000000) [n:127.0.0.1:49410_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 688001 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/cores/collection1
   [junit4]   2> 688002 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001
   [junit4]   2> 688003 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 688008 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f01e7f3{/,null,AVAILABLE}
   [junit4]   2> 688009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@f9a9b1{HTTP/1.1}{127.0.0.1:54578}
   [junit4]   2> 688009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.e.j.s.Server Started @691255ms
   [junit4]   2> 688009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=hdfs://localhost:56115/hdfs__localhost_56115__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001_tempDir-002_jetty1, hostPort=54578, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 688009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 688009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/'
   [junit4]   2> 688032 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 688034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 688035 INFO  (zkCallback-369-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b09c8c7 name:ZooKeeperConnection Watcher:127.0.0.1:44237/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 688035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 688036 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 688038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 688038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/solr.xml
   [junit4]   2> 688062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/cores
   [junit4]   2> 688062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoreContainer New CoreContainer 308061928
   [junit4]   2> 688062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/]
   [junit4]   2> 688062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/lib
   [junit4]   2> 688062 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1C31F1CCECA8AE8-001/shard-1-001/lib).
   [junit4]   2> 688074 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 688076 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 688076 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 688076 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 688077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44237/solr
   [junit4]   2> 688077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 688077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 688077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 688080 INFO  (zkCallback-371-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17cac811 name:ZooKeeperConnection Watcher:127.0.0.1:44237 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 688082 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F1C31F1CCECA8AE8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKee

[...truncated too long message...]

e #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=116935, name=coreZkRegister-6107-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F1C31F1CCECA8AE8]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:422)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3290181 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[F1C31F1CCECA8AE8]-worker) [n:127.0.0.1:43689_wz_thu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_F1C31F1CCECA8AE8-001
   [junit4]   2> Oct 05, 2015 7:26:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 3290454 WARN  (OverseerStateUpdate-94639700595769362-127.0.0.1:43689_wz_thu-n_0000000004) [n:127.0.0.1:43689_wz_thu    ] o.a.s.c.Overseer Solr cannot talk to ZK, exiting Overseer main queue loop
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-0000000041
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.removeFirst(DistributedQueue.java:385)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.poll(DistributedQueue.java:188)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:229)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3290455 INFO  (OverseerStateUpdate-94639700595769362-127.0.0.1:43689_wz_thu-n_0000000004) [n:127.0.0.1:43689_wz_thu    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43689_wz_thu
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=nl_BE, timezone=Antarctica/Mawson
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=128748832,total=526385152
   [junit4]   2> NOTE: All tests run in this JVM: [TestLeaderElectionZkExpiry, StatelessScriptUpdateProcessorFactoryTest, HdfsBasicDistributedZkTest, CloudMLTQParserTest, QueryResultKeyTest, RegexBoostProcessorTest, TestCharFilters, TestNRTOpen, ShowFileRequestHandlerTest, TestSystemIdResolver, TestXIncludeConfig, BasicAuthIntegrationTest, SuggesterTest, TestSmileRequest, TestDistributedMissingSort, NoCacheHeaderTest, TestSolrDeletionPolicy1, TestMiniSolrCloudClusterBase, BJQParserTest, FullSolrCloudDistribCmdsTest, TestLeaderInitiatedRecoveryThread, TestAnalyzeInfixSuggestions, TestSolrConfigHandlerCloud, DirectUpdateHandlerTest, OverseerTest, ChangedSchemaMergeTest, SmileWriterTest, TestSchemaVersionResource, ActionThrottleTest, CoreAdminCreateDiscoverTest, TestSolrQueryParser, FieldMutatingUpdateProcessorTest, PluginInfoTest, MergeStrategyTest, TestSolrCloudWithKerberosAlt, TestStressReorder, SampleTest, DirectSolrConnectionTest, TestShortCircuitedRequests, FacetPivotSmallTest, TestSortingResponseWriter, ZkStateWriterTest, TestRecovery, TestReplicationHandlerBackup, DistributedSuggestComponentTest, ScriptEngineTest, TestSolrDynamicMBean, TestNonDefinedSimilarityFactory, BitVectorTest, TestIndexSearcher, TestDownShardTolerantSearch, TestJmxIntegration, TestExtendedDismaxParser, TestCopyFieldCollectionResource, TestSolrQueryParserDefaultOperatorResource, TestHashQParserPlugin, RequestHandlersTest, RulesTest, LeaderElectionTest, SimpleFacetsTest, TestExactSharedStatsCache, TestFastWriter, TestSchemaManager, SolrTestCaseJ4Test, TestCollapseQParserPlugin, SpatialHeatmapFacetsTest, TestFileDictionaryLookup, DebugComponentTest, UUIDFieldTest, TestDefaultStatsCache, AutoCommitTest, TestSolrQueryParserResource, TestFastLRUCache, TestRemoveLastDynamicCopyField, TestReload, TestJettySolrRunner, TestMinMaxOnMultiValuedField, StatsComponentTest, TestPostingsSolrHighlighter, BadComponentTest, HdfsLockFactoryTest, DistributedQueryComponentCustomSortTest, TestDynamicLoading, TestStressRecovery, ResponseHeaderTest, TestDynamicFieldResource, HdfsDirectoryFactoryTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, LeaderElectionIntegrationTest, RecoveryZkTest, TestReplicationHandler, SolrCoreTest, PeerSyncTest, ConvertedLegacyTest, TestFiltering, CurrencyFieldXmlFileTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, SuggesterTSTTest, TestCSVLoader, SolrCoreCheckLockOnStartupTest, DistanceFunctionTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, PrimitiveFieldTypeTest, DocumentBuilderTest, RequiredFieldsTest, FastVectorHighlighterTest, LoggingHandlerTest, UpdateRequestProcessorFactoryTest, QueryParsingTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, MultiTermTest, NumericFieldsTest, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, SynonymTokenizerTest, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TimeZoneUtilsTest, ResourceLoaderTest, DateMathParserTest, DateFieldTest, CircularListTest, TestRTGBase, TestHighlightDedupGrouping, TestTolerantSearch, TestEmbeddedSolrServerConstructors, AssignTest, AsyncCallRequestStatusResponseTest, CloudExitableDirectoryReaderTest, CollectionTooManyReplicasTest, DistributedQueueTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [434/540] on J2 in 362.40s, 1 test, 1 error <<< FAILURES!

[...truncated 368 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 540 suites (7 ignored), 2129 tests, 3 errors, 29 ignored (17 assumptions) [seed: F1C31F1CCECA8AE8]

Total time: 192 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 976 - Still Failing

Posted by Mark Miller <ma...@gmail.com>.
Hmm...perhaps we don't want this assertion. Need to look closer.

- mark

On Sun, Oct 4, 2015 at 9:31 PM Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/976/
>
> 1 tests failed.
> FAILED:
> org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR
>
> Error Message:
> Captured an uncaught exception in thread: Thread[id=20681,
> name=coreZkRegister-5424-thread-2, state=RUNNABLE,
> group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=20681,
> name=coreZkRegister-5424-thread-2, state=RUNNABLE,
> group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
> Caused by: java.lang.AssertionError
>         at __randomizedtesting.SeedInfo.seed([D95FA07FAB9FF725]:0)
>         at
> org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
>         at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:422)
>         at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
>         at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
>         at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
>         at
> org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
>         at
> org.apache.solr.cloud.ZkController.register(ZkController.java:926)
>         at
> org.apache.solr.cloud.ZkController.register(ZkController.java:881)
>         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
>         at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 10821 lines...]
>    [junit4] JVM J2: stdout was not empty, see:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151004_195707_770.sysout
>    [junit4] >>> JVM J2: stdout (verbatim) ----
>    [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
>    [junit4] Dumping heap to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid8475.hprof
> ...
>    [junit4] Heap dump file created [612713837 bytes in 13.982 secs]
>    [junit4] <<< JVM J2: EOF ----
>
>    [junit4] JVM J2: stderr was not empty, see:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151004_195707_770.syserr
>    [junit4] >>> JVM J2: stderr (verbatim) ----
>    [junit4]
>    [junit4] Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "collection4"
>    [junit4]
>    [junit4] Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Scheduler-963185029"
>    [junit4] WARN: Unhandled exception in event serialization. ->
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>    [junit4] <<< JVM J2: EOF ----
>
> [...truncated 716 lines...]
>    [junit4] Suite:
> org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
>    [junit4]   2> Creating dataDir:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/init-core-data-001
>    [junit4]   2> 3621417 INFO
> (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[D95FA07FAB9FF725]-worker)
> [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system
> property: /_js/
>    [junit4]   2> 3621438 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
>    [junit4]   2> 3621451 INFO  (Thread-12117) [    ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
>    [junit4]   2> 3621451 INFO  (Thread-12117) [    ] o.a.s.c.ZkTestServer
> Starting server
>    [junit4]   2> 3621551 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.ZkTestServer start zk server on port:34088
>    [junit4]   2> 3621551 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3621558 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to
> ZooKeeper
>    [junit4]   2> 3621569 INFO  (zkCallback-1783-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@4587f8fc
> name:ZooKeeperConnection Watcher:127.0.0.1:34088 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3621569 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3621569 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3621569 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /solr
>    [junit4]   2> 3621573 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3621574 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to
> ZooKeeper
>    [junit4]   2> 3621575 INFO  (zkCallback-1784-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@79d4d803
> name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3621575 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3621576 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3621576 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
>    [junit4]   2> 3621578 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
>    [junit4]   2> 3621579 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
>    [junit4]   2> 3621581 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath:
> /collections/control_collection/shards
>    [junit4]   2> 3621582 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
> to /configs/conf1/solrconfig.xml
>    [junit4]   2> 3621583 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
>    [junit4]   2> 3621585 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
> to /configs/conf1/schema.xml
>    [junit4]   2> 3621585 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
>    [junit4]   2> 3621587 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
> to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 3621587 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 3621589 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
> to /configs/conf1/stopwords.txt
>    [junit4]   2> 3621589 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
>    [junit4]   2> 3621591 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt
> to /configs/conf1/protwords.txt
>    [junit4]   2> 3621591 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
>    [junit4]   2> 3621593 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml
> to /configs/conf1/currency.xml
>    [junit4]   2> 3621593 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
>    [junit4]   2> 3621594 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
> to /configs/conf1/enumsConfig.xml
>    [junit4]   2> 3621595 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
>    [junit4]   2> 3621596 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
> to /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 3621596 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 3621598 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
> to /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 3621598 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 3621599 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
> to /configs/conf1/old_synonyms.txt
>    [junit4]   2> 3621600 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
>    [junit4]   2> 3621601 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
> to /configs/conf1/synonyms.txt
>    [junit4]   2> 3621601 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
>    [junit4]   2> 3621888 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1
>    [junit4]   2> 3621890 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.Server jetty-9.2.13.v20150730
>    [junit4]   2> 3621891 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.h.ContextHandler Started
> o.e.j.s.ServletContextHandler@214c3980{/_js,null,AVAILABLE}
>    [junit4]   2> 3621892 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.ServerConnector Started ServerConnector@13531394{HTTP/1.1}{
> 127.0.0.1:35830}
>    [junit4]   2> 3621892 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.Server Started @3624951ms
>    [junit4]   2> 3621892 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties:
> {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/tempDir-001/control/data,
> hostContext=/_js,
> coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores,
> hostPort=35830}
>    [junit4]   2> 3621892 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
> sun.misc.Launcher$AppClassLoader@1fe91485
>    [junit4]   2> 3621892 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/'
>    [junit4]   2> 3621908 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3621909 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to
> ZooKeeper
>    [junit4]   2> 3621910 INFO  (zkCallback-1785-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@431ab8f6
> name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3621910 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3621911 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3621924 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found
> in ZooKeeper)
>    [junit4]   2> 3621924 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.SolrXmlConfig Loading container configuration from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/solr.xml
>    [junit4]   2> 3621938 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoresLocator Config-defined core root directory:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores
>    [junit4]   2> 3621939 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoreContainer New CoreContainer 826014353
>    [junit4]   2> 3621939 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer
> [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/]
>    [junit4]   2> 3621939 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoreContainer loading shared library:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/lib
>    [junit4]   2> 3621939 WARN
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to
> classloader: lib (resolved as:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/lib).
>    [junit4]   2> 3621947 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout :
> 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost :
> 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize :
> 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy :
> false,useRetries : false,
>    [junit4]   2> 3621948 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client
> with params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 3621949 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 3621949 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j
> (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 3621949 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34088/solr
>    [junit4]   2> 3621949 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 3621949 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3621950 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to
> ZooKeeper
>    [junit4]   2> 3621978 INFO  (zkCallback-1787-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@37f6f361
> name:ZooKeeperConnection Watcher:127.0.0.1:34088 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3621979 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3621979 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3621981 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.ConnectionManager Waiting for
> client to connect to ZooKeeper
>    [junit4]   2> 3621984 INFO
> (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@5d1226bd
> name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3621984 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.ConnectionManager Client is
> connected to ZooKeeper
>    [junit4]   2> 3621985 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/queue
>    [junit4]   2> 3621987 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-queue-work
>    [junit4]   2> 3621990 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-running
>    [junit4]   2> 3621992 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-completed
>    [junit4]   2> 3621993 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-failure
>    [junit4]   2> 3621995 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
>    [junit4]   2> 3621996 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
>    [junit4]   2> 3621997 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /clusterstate.json
>    [junit4]   2> 3621998 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /security.json
>    [junit4]   2> 3621999 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.ZkController Register node as live in
> ZooKeeper:/live_nodes/127.0.0.1:35830__js
>    [junit4]   2> 3621999 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /live_nodes/127.0.0.1:35830__js
>    [junit4]   2> 3622000 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect
>    [junit4]   2> 3622001 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect/election
>    [junit4]   2> 3622002 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 3622002 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.LeaderElector Joined leadership
> election with path:
> /overseer_elect/election/94634419975290884-127.0.0.1:35830__js-n_0000000000
>    [junit4]   2> 3622003 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.OverseerElectionContext I am going to
> be the leader 127.0.0.1:35830__js
>    [junit4]   2> 3622003 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect/leader
>    [junit4]   2> 3622004 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer Overseer
> (id=94634419975290884-127.0.0.1:35830__js-n_0000000000) starting
>    [junit4]   2> 3622005 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/queue-work
>    [junit4]   2> 3622010 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.OverseerAutoReplicaFailoverThread
> Starting OverseerAutoReplicaFailoverThread
> autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
>    [junit4]   2> 3622013 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 3622013 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer Starting to work on the main
> queue
>    [junit4]   2> 3622016 INFO
> (OverseerCollectionConfigSetProcessor-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.OverseerTaskProcessor Process current
> queue of overseer operations
>    [junit4]   2> 3622048 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.CoreContainer Security conf doesn't
> exist. Skipping setup for authorization module.
>    [junit4]   2> 3622048 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.CoreContainer No authentication
> plugin used.
>    [junit4]   2> 3622049 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.CoresLocator Looking for core
> definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores
>    [junit4]   2> 3622050 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.SolrCore Created CoreDescriptor:
> {name=collection1, shard=, config=solrconfig.xml,
> configSetProperties=configsetprops.json, loadOnStartup=true,
> transient=false,
> instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1,
> absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/,
> collection=control_collection, schema=schema.xml, dataDir=data/,
> coreNodeName=}
>    [junit4]   2> 3622050 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.CoresLocator Found core collection1
> in
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/
>    [junit4]   2> 3622050 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.c.CoresLocator Found 1 core definitions
>    [junit4]   2> 3622056 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.s.SolrDispatchFilter
> user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
>    [junit4]   2> 3622056 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [n:127.0.0.1:35830__js    ] o.a.s.s.SolrDispatchFilter
> SolrDispatchFilter.init() done
>    [junit4]   2> 3622060 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.ZkController publishing state=down
>    [junit4]   2> 3622060 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.ZkController numShards not found on descriptor - reading it from
> system property
>    [junit4]   2> 3622061 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize:
> 1, message = {
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "node_name":"127.0.0.1:35830__js",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "roles":null} current state version: 0
>    [junit4]   2> 3622061 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.ZkController look for our core node name
>    [junit4]   2> 3622062 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Update state
> numShards=1 message={
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "node_name":"127.0.0.1:35830__js",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "roles":null}
>    [junit4]   2> 3622062 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.o.ClusterStateMutator building a new
> cName: control_collection
>    [junit4]   2> 3622062 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Assigning new node
> to shard shard=shard1
>    [junit4]   2> 3622064 INFO
> (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 3623062 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.ZkController waiting to find shard id in clusterstate for
> collection1
>    [junit4]   2> 3623062 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.ZkController Check for collection zkNode:control_collection
>    [junit4]   2> 3623063 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.ZkController Collection zkNode exists
>    [junit4]   2> 3623063 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.c.ZkStateReader Load collection config
> from:/collections/control_collection
>    [junit4]   2> 3623063 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.c.ZkStateReader path=/collections/control_collection
> configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 3623063 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/'
>    [junit4]   2> 3623079 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.Config loaded config solrconfig.xml with version 0
>    [junit4]   2> 3623084 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.SolrConfig current version of requestparams : -1
>    [junit4]   2> 3623097 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
>    [junit4]   2> 3623137 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
>    [junit4]   2> 3623137 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
>    [junit4]   2> 3623144 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.IndexSchema [collection1] Schema name=test
>    [junit4]   2> 3623396 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.OpenExchangeRatesOrgProvider Initialized with
> rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 3623412 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.IndexSchema default search field in schema is text
>    [junit4]   2> 3623414 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.IndexSchema unique key field: id
>    [junit4]   2> 3623423 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file
> currency.xml
>    [junit4]   2> 3623426 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file
> currency.xml
>    [junit4]   2> 3623428 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from
> open-exchange-rates.json
>    [junit4]   2> 3623429 WARN
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
>    [junit4]   2> 3623429 WARN
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
>    [junit4]   2> 3623429 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from
> open-exchange-rates.json
>    [junit4]   2> 3623429 WARN
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
>    [junit4]   2> 3623429 WARN
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
>    [junit4]   2> 3623430 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming
> default properties: Can't find resource 'configsetprops.json' in classpath
> or '/configs/conf1',
> cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
>    [junit4]   2> 3623430 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection   x:collection1]
> o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration
> from collection control_collection
>    [junit4]   2> 3623431 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
>    [junit4]   2> 3623431 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at
> [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/],
> dataDir=[null]
>    [junit4]   2> 3623431 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding
> Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7336c25
>    [junit4]   2> 3623431 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data
>    [junit4]   2> 3623431 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore New index directory detected: old=null
> new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/index/
>    [junit4]   2> 3623432 WARN
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/index'
> doesn't exist. Creating new index...
>    [junit4]   2> 3623432 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/index
>    [junit4]   2> 3623432 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy:
> minMergeSize=1000, mergeFactor=15, maxMergeSize=9223372036854775807,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.11632999180321271]
>    [junit4]   2> 3623432 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
>    [junit4]   2>
> commit{dir=MockDirectoryWrapper(RAMDirectory@d477eab
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@50cd135c
> ),segFN=segments_1,generation=1}
>    [junit4]   2> 3623432 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore newest commit generation = 1
>    [junit4]   2> 3623433 INFO
> (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore
> Looking for old index directories to cleanup for core collection1 in
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/
>    [junit4]   2> 3623433 WARN
> (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.DirectoryFactory
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/
> does not point to a valid data directory; skipping clean-up of old index
> directories.
>    [junit4]   2> 3623437 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 3623437 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "dedupe"
>    [junit4]   2> 3623438 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
> DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
>    [junit4]   2> 3623438 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 3623438 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
> DistributedUpdateProcessorFactory into updateRequestProcessorChain
> "stored_sig"
>    [junit4]   2> 3623438 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 3623438 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 3623439 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
> DistributedUpdateProcessorFactory into updateRequestProcessorChain
> "distrib-dup-test-chain-implicit"
>    [junit4]   2> 3623439 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as
> default, creating implicit default
>    [junit4]   2> 3623441 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3623443 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3623444 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3623445 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3623448 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.RequestHandlers Registered paths:
> /admin/segments,/admin/threads,/admin/luke,/admin/properties,/get,/admin/ping,/admin/logging,/update/json,/admin/mbeans,/update/json/docs,/admin/file,/admin/plugins,/update/csv,/replication,/schema,standard,/config,/update,/admin/system
>    [junit4]   2> 3623449 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore Using default statsCache cache:
> org.apache.solr.search.stats.LocalStatsCache
>    [junit4]   2> 3623450 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation:
> org.apache.solr.update.UpdateLog
>    [junit4]   2> 3623450 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=
> defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
> numVersionBuckets=65536
>    [junit4]   2> 3623451 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
>    [junit4]   2> 3623451 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
>    [junit4]   2> 3623451 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy:
> minMergeSize=1000, mergeFactor=35, maxMergeSize=9223372036854775807,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
>    [junit4]   2> 3623451 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
>    [junit4]   2>
> commit{dir=MockDirectoryWrapper(RAMDirectory@d477eab
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@50cd135c
> ),segFN=segments_1,generation=1}
>    [junit4]   2> 3623451 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore newest commit generation = 1
>    [junit4]   2> 3623452 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6209a64f[collection1]
> main
>    [junit4]   2> 3623452 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.c.ZkStateReader Load collection config
> from:/collections/control_collection
>    [junit4]   2> 3623452 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection
> configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 3623452 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based
> storage for the RestManager with znodeBase: /configs/conf1
>    [junit4]   2> 3623453 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO
> with znodeBase: /configs/conf1
>    [junit4]   2> 3623453 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs:
> {}
>    [junit4]   2> 3623453 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json
> using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 3623453 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode
> /configs/conf1/_rest_managed.json
>    [junit4]   2> 3623453 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path
> _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 3623454 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.RestManager Initializing 0 registered
> ManagedResources
>    [junit4]   2> 3623454 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for
> 10000
>    [junit4]   2> 3623454 INFO
> (searcherExecutor-5387-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore
> [collection1] Registered new searcher Searcher@6209a64f[collection1]
> main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>    [junit4]   2> 3623454 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to
> seed version buckets
>    [junit4]   2> 3623454 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_
> for 65536 version buckets from index
>    [junit4]   2> 3623455 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot
> seed version bucket highest value from index
>    [junit4]   2> 3623455 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateLog Could not find max version in index or
> recent updates, using new clock 1514150721689223168
>    [junit4]   2> 3623457 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with
> highest version 1514150721689223168
>    [junit4]   2> 3623457 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
>    [junit4]   2> 3623457 INFO
> (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.CoreContainer registering core: collection1
>    [junit4]   2> 3623458 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ZkController Register replica - core:collection1 address:
> http://127.0.0.1:35830/_js collection:control_collection shard:shard1
>    [junit4]   2> 3623458 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/control_collection/leader_elect/shard1/election
>    [junit4]   2> 3623461 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/control_collection/leader_elect/shard1/election/94634419975290884-core_node1-n_0000000000
>    [junit4]   2> 3623462 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext Running the leader process for shard
> shard1
>    [junit4]   2> 3623463 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
>    [junit4]   2> 3623463 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
>    [junit4]   2> 3623463 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.SyncStrategy Sync replicas to
> http://127.0.0.1:35830/_js/collection1/
>    [junit4]   2> 3623464 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize:
> 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection"} current state
> version: 1
>    [junit4]   2> 3623464 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
>    [junit4]   2> 3623464 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.SyncStrategy http://127.0.0.1:35830/_js/collection1/ has no
> replicas
>    [junit4]   2> 3623464 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext I am the new leader:
> http://127.0.0.1:35830/_js/collection1/ shard1
>    [junit4]   2> 3623464 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/control_collection/leaders/shard1
>    [junit4]   2> 3623466 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
>    [junit4]   2> 3623469 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize:
> 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "state":"active"} current state version: 1
>    [junit4]   2> 3623558 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3623558 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to
> ZooKeeper
>    [junit4]   2> 3623560 INFO  (zkCallback-1790-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@126f73d2
> name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3623560 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3623560 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3623560 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 3623562 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause
> connection loss:false
>    [junit4]   2> 3623562 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with
> stateFormat=2
>    [junit4]   2> 3623562 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3623563 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to
> ZooKeeper
>    [junit4]   2> 3623564 INFO  (zkCallback-1791-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@357c9f02
> name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3623565 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3623565 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3623567 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize:
> 1, message = {
>    [junit4]   2>   "operation":"create",
>    [junit4]   2>   "name":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "stateFormat":"2"} current state version: 1
>    [junit4]   2> 3623567 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.o.ClusterStateMutator building a new
> cName: collection1
>    [junit4]   2> 3623569 INFO
> (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 3623569 INFO  (zkCallback-1790-thread-1) [    ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
>    [junit4]   2> 3623620 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ZkController We are http://127.0.0.1:35830/_js/collection1/ and
> leader is http://127.0.0.1:35830/_js/collection1/
>    [junit4]   2> 3623620 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=
> http://127.0.0.1:35830/_js
>    [junit4]   2> 3623620 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ZkController I am the leader, no recovery necessary
>    [junit4]   2> 3623620 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ZkController publishing state=active
>    [junit4]   2> 3623620 INFO
> (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js
> c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ZkController numShards not found on descriptor - reading it from
> system property
>    [junit4]   2> 3623622 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize:
> 1, message = {
>    [junit4]   2>   "state":"active",
>    [junit4]   2>   "node_name":"127.0.0.1:35830__js",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "core_node_name":"core_node1",
>    [junit4]   2>   "roles":null} current state version: 2
>    [junit4]   2> 3623623 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Update state
> numShards=1 message={
>    [junit4]   2>   "state":"active",
>    [junit4]   2>   "node_name":"127.0.0.1:35830__js",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "core_node_name":"core_node1",
>    [junit4]   2>   "roles":null}
>    [junit4]   2> 3623623 INFO
> (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000)
> [n:127.0.0.1:35830__js    ] o.a.s.c.o.ZkStateWriter going to
> create_collection /collections/collection1/state.json
>    [junit4]   2> 3623727 INFO  (zkCallback-1790-thread-1) [    ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
>    [junit4]   2> 3623727 INFO
> (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js)
> [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 3623861 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores/collection1
>    [junit4]   2> 3623862 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001
>    [junit4]   2> 3623862 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.Server jetty-9.2.13.v20150730
>    [junit4]   2> 3623864 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@618274
> {/_js,null,AVAILABLE}
>    [junit4]   2> 3623864 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.ServerConnector Started ServerConnector@405eeab5{HTTP/1.1}{
> 127.0.0.1:53915}
>    [junit4]   2> 3623864 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.e.j.s.Server Started @3626923ms
>    [junit4]   2> 3623864 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties:
> {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores,
> hostPort=53915, solrconfig=solrconfig.xml,
> solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/tempDir-001/jetty1,
> hostContext=/_js}
>    [junit4]   2> 3623865 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
> sun.misc.Launcher$AppClassLoader@1fe91485
>    [junit4]   2> 3623865 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/'
>    [junit4]   2> 3623880 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3623880 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to
> ZooKeeper
>    [junit4]   2> 3623882 INFO  (zkCallback-1792-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6c500b5a
> name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3623882 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3623882 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3623883 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found
> in ZooKeeper)
>    [junit4]   2> 3623883 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.SolrXmlConfig Loading container configuration from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/solr.xml
>    [junit4]   2> 3623897 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoresLocator Config-defined core root directory:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores
>    [junit4]   2> 3623897 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoreContainer New CoreContainer 622281557
>    [junit4]   2> 3623897 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer
> [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/]
>    [junit4]   2> 3623897 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.CoreContainer loading shared library:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/lib
>    [junit4]   2> 3623897 WARN
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to
> classloader: lib (resolved as:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/lib).
>    [junit4]   2> 3623905 INFO
> (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725])
> [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout :
> 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost :
> 20,maxConnections : 10000,core

-- 
- Mark
about.me/markrmiller

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 976 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/976/

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=20681, name=coreZkRegister-5424-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20681, name=coreZkRegister-5424-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([D95FA07FAB9FF725]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:422)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10821 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151004_195707_770.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid8475.hprof ...
   [junit4] Heap dump file created [612713837 bytes in 13.982 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151004_195707_770.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "collection4"
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Scheduler-963185029"
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J2: EOF ----

[...truncated 716 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/init-core-data-001
   [junit4]   2> 3621417 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[D95FA07FAB9FF725]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_js/
   [junit4]   2> 3621438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3621451 INFO  (Thread-12117) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3621451 INFO  (Thread-12117) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3621551 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.ZkTestServer start zk server on port:34088
   [junit4]   2> 3621551 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3621558 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3621569 INFO  (zkCallback-1783-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4587f8fc name:ZooKeeperConnection Watcher:127.0.0.1:34088 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3621569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3621569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3621569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3621573 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3621574 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3621575 INFO  (zkCallback-1784-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79d4d803 name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3621575 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3621576 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3621576 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3621578 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3621579 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3621581 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3621582 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3621583 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3621585 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3621585 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3621587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3621587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3621589 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3621589 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3621591 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3621591 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3621593 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3621593 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3621594 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3621595 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3621596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3621596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3621598 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3621598 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3621599 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3621600 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3621601 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3621601 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3621888 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1
   [junit4]   2> 3621890 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3621891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@214c3980{/_js,null,AVAILABLE}
   [junit4]   2> 3621892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.ServerConnector Started ServerConnector@13531394{HTTP/1.1}{127.0.0.1:35830}
   [junit4]   2> 3621892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.Server Started @3624951ms
   [junit4]   2> 3621892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/tempDir-001/control/data, hostContext=/_js, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores, hostPort=35830}
   [junit4]   2> 3621892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 3621892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/'
   [junit4]   2> 3621908 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3621909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3621910 INFO  (zkCallback-1785-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@431ab8f6 name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3621910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3621911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3621924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3621924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/solr.xml
   [junit4]   2> 3621938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores
   [junit4]   2> 3621939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoreContainer New CoreContainer 826014353
   [junit4]   2> 3621939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/]
   [junit4]   2> 3621939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/lib
   [junit4]   2> 3621939 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/lib).
   [junit4]   2> 3621947 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3621948 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3621949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3621949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3621949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34088/solr
   [junit4]   2> 3621949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3621949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3621950 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3621978 INFO  (zkCallback-1787-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37f6f361 name:ZooKeeperConnection Watcher:127.0.0.1:34088 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3621979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3621979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3621981 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3621984 INFO  (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d1226bd name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3621984 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3621985 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3621987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3621990 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3621992 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3621993 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3621995 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3621996 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3621997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3621998 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3621999 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35830__js
   [junit4]   2> 3621999 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35830__js
   [junit4]   2> 3622000 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3622001 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3622002 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3622002 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94634419975290884-127.0.0.1:35830__js-n_0000000000
   [junit4]   2> 3622003 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35830__js
   [junit4]   2> 3622003 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3622004 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer Overseer (id=94634419975290884-127.0.0.1:35830__js-n_0000000000) starting
   [junit4]   2> 3622005 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3622010 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3622013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3622013 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3622016 INFO  (OverseerCollectionConfigSetProcessor-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3622048 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3622048 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3622049 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores
   [junit4]   2> 3622050 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, shard=, config=solrconfig.xml, configSetProperties=configsetprops.json, loadOnStartup=true, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/, collection=control_collection, schema=schema.xml, dataDir=data/, coreNodeName=}
   [junit4]   2> 3622050 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/
   [junit4]   2> 3622050 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3622056 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3622056 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:35830__js    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3622060 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3622060 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3622061 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35830__js",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 3622061 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3622062 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35830__js",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3622062 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3622062 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3622064 INFO  (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3623062 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3623062 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3623063 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3623063 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3623063 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3623063 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/'
   [junit4]   2> 3623079 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3623084 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3623097 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3623137 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3623137 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3623144 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3623396 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3623412 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3623414 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3623423 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3623426 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3623428 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3623429 WARN  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3623429 WARN  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3623429 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3623429 WARN  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3623429 WARN  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3623430 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3623430 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3623431 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3623431 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3623431 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7336c25
   [junit4]   2> 3623431 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data
   [junit4]   2> 3623431 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3623432 WARN  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3623432 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/index
   [junit4]   2> 3623432 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=15, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.11632999180321271]
   [junit4]   2> 3623432 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@d477eab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@50cd135c),segFN=segments_1,generation=1}
   [junit4]   2> 3623432 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3623433 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/
   [junit4]   2> 3623433 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 3623437 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3623437 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3623438 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3623438 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3623438 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3623438 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3623438 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3623439 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3623439 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3623441 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3623443 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3623444 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3623445 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3623448 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/admin/threads,/admin/luke,/admin/properties,/get,/admin/ping,/admin/logging,/update/json,/admin/mbeans,/update/json/docs,/admin/file,/admin/plugins,/update/csv,/replication,/schema,standard,/config,/update,/admin/system
   [junit4]   2> 3623449 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3623450 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3623450 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3623451 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3623451 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3623451 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=35, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3623451 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@d477eab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@50cd135c),segFN=segments_1,generation=1}
   [junit4]   2> 3623451 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3623452 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6209a64f[collection1] main
   [junit4]   2> 3623452 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3623452 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3623452 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3623453 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3623453 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3623453 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3623453 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3623453 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3623454 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3623454 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3623454 INFO  (searcherExecutor-5387-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6209a64f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3623454 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3623454 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3623455 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3623455 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514150721689223168
   [junit4]   2> 3623457 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514150721689223168
   [junit4]   2> 3623457 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3623457 INFO  (coreLoadExecutor-5386-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3623458 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:35830/_js collection:control_collection shard:shard1
   [junit4]   2> 3623458 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3623461 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94634419975290884-core_node1-n_0000000000
   [junit4]   2> 3623462 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3623463 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3623463 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3623463 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35830/_js/collection1/
   [junit4]   2> 3623464 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3623464 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3623464 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35830/_js/collection1/ has no replicas
   [junit4]   2> 3623464 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35830/_js/collection1/ shard1
   [junit4]   2> 3623464 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3623466 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3623469 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3623558 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3623558 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3623560 INFO  (zkCallback-1790-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@126f73d2 name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3623560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3623560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3623560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3623562 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3623562 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3623562 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3623563 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3623564 INFO  (zkCallback-1791-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@357c9f02 name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3623565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3623565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3623567 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 3623567 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3623569 INFO  (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3623569 INFO  (zkCallback-1790-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3623620 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:35830/_js/collection1/ and leader is http://127.0.0.1:35830/_js/collection1/
   [junit4]   2> 3623620 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35830/_js
   [junit4]   2> 3623620 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3623620 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3623620 INFO  (coreZkRegister-5380-thread-1-processing-r:core_node1 n:127.0.0.1:35830__js c:control_collection x:collection1 s:shard1) [n:127.0.0.1:35830__js c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3623622 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:35830__js",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null} current state version: 2
   [junit4]   2> 3623623 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:35830__js",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:35830/_js",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3623623 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3623727 INFO  (zkCallback-1790-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3623727 INFO  (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3623861 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores/collection1
   [junit4]   2> 3623862 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001
   [junit4]   2> 3623862 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3623864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@618274{/_js,null,AVAILABLE}
   [junit4]   2> 3623864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.ServerConnector Started ServerConnector@405eeab5{HTTP/1.1}{127.0.0.1:53915}
   [junit4]   2> 3623864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.e.j.s.Server Started @3626923ms
   [junit4]   2> 3623864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores, hostPort=53915, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/tempDir-001/jetty1, hostContext=/_js}
   [junit4]   2> 3623865 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 3623865 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/'
   [junit4]   2> 3623880 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3623880 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3623882 INFO  (zkCallback-1792-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c500b5a name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3623882 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3623882 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3623883 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3623883 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/solr.xml
   [junit4]   2> 3623897 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores
   [junit4]   2> 3623897 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoreContainer New CoreContainer 622281557
   [junit4]   2> 3623897 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/]
   [junit4]   2> 3623897 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/lib
   [junit4]   2> 3623897 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/lib).
   [junit4]   2> 3623905 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3623906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3623906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3623906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3623906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34088/solr
   [junit4]   2> 3623907 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3623907 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3623907 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3623909 INFO  (zkCallback-1794-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59611c66 name:ZooKeeperConnection Watcher:127.0.0.1:34088 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3623909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3623910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3623911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3623912 INFO  (zkCallback-1795-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11256069 name:ZooKeeperConnection Watcher:127.0.0.1:34088/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3623913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3623918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3624924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53915__js
   [junit4]   2> 3624924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53915__js
   [junit4]   2> 3624926 INFO  (zkCallback-1788-thread-1-processing-n:127.0.0.1:35830__js) [n:127.0.0.1:35830__js    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3624926 INFO  (zkCallback-1790-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3624926 INFO  (zkCallback-1795-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3624927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3624928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94634419975290889-127.0.0.1:53915__js-n_0000000001
   [junit4]   2> 3624928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94634419975290884-127.0.0.1:35830__js-n_0000000000 to know if I could be the leader
   [junit4]   2> 3624949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3624949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3624950 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores
   [junit4]   2> 3624951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores/collection1, coreNodeName=, config=solrconfig.xml, configSetProperties=configsetprops.json, dataDir=data/, shard=, schema=schema.xml, name=collection1, loadOnStartup=true, collection=collection1, transient=false}
   [junit4]   2> 3624951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores/collection1/
   [junit4]   2> 3624951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3624952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3624952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[D95FA07FAB9FF725]) [n:127.0.0.1:53915__js    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3624952 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3624953 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3624954 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 3624954 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 3624954 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 3624954 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:53915__js",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:53915/_js",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null} current state version: 3
   [junit4]   2> 3624954 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3624955 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:53915__js",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:53915/_js",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3624955 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 3624955 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3625057 INFO  (OverseerStateUpdate-94634419975290884-127.0.0.1:35830__js-n_0000000000) [n:127.0.0.1:35830__js    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 3625058 INFO  (zkCallback-1795-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3625059 INFO  (zkCallback-1795-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 3625955 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3625955 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 3625955 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3625955 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 3625956 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3625956 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_D95FA07FAB9FF725-001/shard-1-001/cores/collection1/'
   [junit4]   2> 3625971 INFO  (coreLoadExecutor-5397-thread-1-processing-n:127.0.0.1:53915__js) [n:127.0.0.1:53915__js c:collection1   x:collection1] o.a.s.c.Config loaded conf

[...truncated too long message...]

n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.17.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151004_195707_770.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151004_195707_770.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1505)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 317 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 975 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/975/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2870, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2870, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44582: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([5DE68C2A41FF7D9B]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=21532, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21532, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([5DE68C2A41FF7D9B:D5B2B3F0EF031063]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59364: Could not find collection : awholynewstresscollection_collection1_2
	at __randomizedtesting.SeedInfo.seed([5DE68C2A41FF7D9B]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10254 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/init-core-data-001
   [junit4]   2> 293624 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[5DE68C2A41FF7D9B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 293624 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[5DE68C2A41FF7D9B]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 293628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 293629 INFO  (Thread-1385) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 293629 INFO  (Thread-1385) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 293729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.ZkTestServer start zk server on port:50996
   [junit4]   2> 293729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 293730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 293732 INFO  (zkCallback-252-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@13cef6c4 name:ZooKeeperConnection Watcher:127.0.0.1:50996 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 293733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 293733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 293733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 293736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 293736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 293738 INFO  (zkCallback-253-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f752f6f name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 293738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 293738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 293739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 293741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 293743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 293744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 293746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 293747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 293750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 293750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 293753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 293753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 293755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 293755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 293757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 293757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 293759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 293759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 293761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 293761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 293764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 293764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 293766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 293766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 293768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 293768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 293770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 293770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 293773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 293774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 293775 INFO  (zkCallback-254-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44f23f63 name:ZooKeeperConnection Watcher:127.0.0.1:50996 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 293775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 293775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 293775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 293778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 293778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 293779 INFO  (zkCallback-255-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@663bdb36 name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 293779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 293780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 293780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 293780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 293783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 293783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 293785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 293785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 293787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 293787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 293789 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 293789 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 293792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 293792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 293794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 293794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 293796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 293796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 293798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 293798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 293800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 293800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 293802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 293802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 293804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 293804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 294130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1
   [junit4]   2> 294132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 294135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3385e47a{/,null,AVAILABLE}
   [junit4]   2> 294143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1b5b2003{HTTP/1.1}{127.0.0.1:51067}
   [junit4]   2> 294143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.Server Started @297299ms
   [junit4]   2> 294143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=51067, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/tempDir-001/control/data}
   [junit4]   2> 294144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 294144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/'
   [junit4]   2> 294164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 294164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 294166 INFO  (zkCallback-256-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c80d0e5 name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 294166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 294166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 294168 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150303f2c9d0004, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 294168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 294169 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/solr.xml
   [junit4]   2> 294188 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores
   [junit4]   2> 294189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoreContainer New CoreContainer 1285472978
   [junit4]   2> 294189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/]
   [junit4]   2> 294189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/lib
   [junit4]   2> 294189 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/lib).
   [junit4]   2> 294198 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 294200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 294200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 294200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 294200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50996/solr
   [junit4]   2> 294200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 294200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 294201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 294203 INFO  (zkCallback-258-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b82f7f6 name:ZooKeeperConnection Watcher:127.0.0.1:50996 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 294203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 294204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 294206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 294207 INFO  (zkCallback-259-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b448545 name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 294207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 294209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 294212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 294216 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 294218 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 294221 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 294225 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 294226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 294228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 294229 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 294231 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51067_
   [junit4]   2> 294232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51067_
   [junit4]   2> 294234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 294235 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 294237 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 294237 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94628640064339974-127.0.0.1:51067_-n_0000000000
   [junit4]   2> 294238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51067_
   [junit4]   2> 294238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 294240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer Overseer (id=94628640064339974-127.0.0.1:51067_-n_0000000000) starting
   [junit4]   2> 294242 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 294249 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 294249 INFO  (OverseerCollectionConfigSetProcessor-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 294250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 294250 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 294266 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 294266 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 294267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores
   [junit4]   2> 294268 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, config=solrconfig.xml, configSetProperties=configsetprops.json, dataDir=data/, coreNodeName=, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/, transient=false, schema=schema.xml, name=collection1, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1}
   [junit4]   2> 294268 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/
   [junit4]   2> 294268 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 294276 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 294276 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 294278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 294278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:51067_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 294279 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51067",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51067_"} current state version: 0
   [junit4]   2> 294279 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51067",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51067_"}
   [junit4]   2> 294279 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 294280 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 294280 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 294282 INFO  (zkCallback-259-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 295280 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 295280 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 295280 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 295280 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 295282 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 295282 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/'
   [junit4]   2> 295298 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 295308 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 295327 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 295357 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 295358 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 295366 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 295631 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 295640 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 295642 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 295657 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 295661 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 295664 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 295665 WARN  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 295665 WARN  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 295666 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 295669 WARN  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 295669 WARN  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 295671 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 295671 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 295671 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 295671 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 295672 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c6049b1
   [junit4]   2> 295672 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/data
   [junit4]   2> 295673 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/data/index/
   [junit4]   2> 295673 WARN  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 295673 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/data/index
   [junit4]   2> 295673 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=32.7919921875, floorSegmentMB=0.4287109375, forceMergeDeletesPctAllowed=14.526048804125903, segmentsPerTier=15.0, maxCFSSegmentSizeMB=1.048828125, noCFSRatio=0.363368438613632
   [junit4]   2> 295674 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5e7aaf2c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4882a229),segFN=segments_1,generation=1}
   [junit4]   2> 295674 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 295680 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 295680 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 295680 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 295680 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 295680 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 295681 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 295681 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 295681 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 295681 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 295683 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/data/
   [junit4]   2> 295683 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 295687 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 295689 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 295691 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 295692 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 295698 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/admin/ping,/update/json/docs,/update,/update/csv,/admin/logging,standard,/admin/file,/config,/admin/threads,/get,/admin/luke,/schema,/admin/properties,/update/json,/admin/mbeans,/admin/system,/admin/plugins,/admin/segments
   [junit4]   2> 295700 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 295700 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 295700 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 295701 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 295701 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 295702 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 295702 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5e7aaf2c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4882a229),segFN=segments_1,generation=1}
   [junit4]   2> 295702 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 295702 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@57ddf71c[collection1] main
   [junit4]   2> 295702 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 295704 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 295704 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 295704 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 295704 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 295705 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 295705 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 295705 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 295705 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 295706 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 295706 INFO  (searcherExecutor-694-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@57ddf71c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 295707 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 295707 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 295707 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 295707 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514058243207331840
   [junit4]   2> 295709 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1514058243207331840
   [junit4]   2> 295710 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 295710 INFO  (coreLoadExecutor-693-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 295711 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:51067 collection:control_collection shard:shard1
   [junit4]   2> 295712 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 295718 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94628640064339974-core_node1-n_0000000000
   [junit4]   2> 295718 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 295721 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 295721 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 295721 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51067/collection1/
   [junit4]   2> 295721 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 295721 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51067/collection1/ has no replicas
   [junit4]   2> 295721 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51067/collection1/ shard1
   [junit4]   2> 295722 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 295722 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 295725 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 295728 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51067",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 295780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 295782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 295783 INFO  (zkCallback-261-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7863437a name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 295784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 295784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 295784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 295787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 295787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 295788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 295788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 295789 INFO  (zkCallback-262-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@644d8e61 name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 295789 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 295790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 295793 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 295793 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 295794 INFO  (zkCallback-261-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 295794 INFO  (zkCallback-259-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 295828 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:51067/collection1/ and leader is http://127.0.0.1:51067/collection1/
   [junit4]   2> 295828 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51067
   [junit4]   2> 295828 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 295828 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 295828 INFO  (coreZkRegister-687-thread-1-processing-x:collection1 r:core_node1 c:control_collection n:127.0.0.1:51067_ s:shard1) [n:127.0.0.1:51067_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 295832 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51067",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:51067_"} current state version: 2
   [junit4]   2> 295833 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51067",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:51067_"}
   [junit4]   2> 295834 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 295939 INFO  (zkCallback-259-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 295939 INFO  (zkCallback-261-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 296089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/cores/collection1
   [junit4]   2> 296090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001
   [junit4]   2> 296091 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 296094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@125b28d7{/,null,AVAILABLE}
   [junit4]   2> 296095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.ServerConnector Started ServerConnector@536465e{HTTP/1.1}{127.0.0.1:36941}
   [junit4]   2> 296095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.e.j.s.Server Started @299251ms
   [junit4]   2> 296095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=36941, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/tempDir-001/jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 296095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 296095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/'
   [junit4]   2> 296113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 296114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 296115 INFO  (zkCallback-263-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e67f10d name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 296115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 296115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 296117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 296117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/solr.xml
   [junit4]   2> 296131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/cores
   [junit4]   2> 296132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoreContainer New CoreContainer 555298978
   [junit4]   2> 296132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/]
   [junit4]   2> 296132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/lib
   [junit4]   2> 296132 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/lib).
   [junit4]   2> 296140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 296142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 296142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 296142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 296142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50996/solr
   [junit4]   2> 296142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 296142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 296143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 296145 INFO  (zkCallback-265-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b83d57b name:ZooKeeperConnection Watcher:127.0.0.1:50996 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 296145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 296146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 296147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 296149 INFO  (zkCallback-266-thread-1-processing-n:127.0.0.1:36941_) [n:127.0.0.1:36941_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@319e3e2f name:ZooKeeperConnection Watcher:127.0.0.1:50996/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 296149 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 296159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 297169 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36941_
   [junit4]   2> 297169 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36941_
   [junit4]   2> 297171 INFO  (zkCallback-259-thread-1-processing-n:127.0.0.1:51067_) [n:127.0.0.1:51067_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 297171 INFO  (zkCallback-261-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 297171 INFO  (zkCallback-266-thread-1-processing-n:127.0.0.1:36941_) [n:127.0.0.1:36941_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 297173 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 297173 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94628640064339979-127.0.0.1:36941_-n_0000000001
   [junit4]   2> 297174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94628640064339974-127.0.0.1:51067_-n_0000000000 to know if I could be the leader
   [junit4]   2> 297192 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 297192 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 297193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/cores
   [junit4]   2> 297194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/cores/collection1/, dataDir=data/, collection=collection1, coreNodeName=, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/cores/collection1, configSetProperties=configsetprops.json, schema=schema.xml, transient=false, config=solrconfig.xml, loadOnStartup=true, shard=}
   [junit4]   2> 297194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_5DE68C2A41FF7D9B-001/shard-1-001/cores/collection1/
   [junit4]   2> 297194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 297196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 297197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[5DE68C2A41FF7D9B]) [n:127.0.0.1:36941_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 297198 INFO  (coreLoadExecutor-704-thread-1-processing-n:127.0.0.1:36941_) [n:127.0.0.1:36941_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 297198 INFO  (coreLoadExecutor-704-thread-1-processing-n:127.0.0.1:36941_) [n:127.0.0.1:36941_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 297200 INFO  (coreLoadExecutor-704-thread-1-processing-n:127.0.0.1:36941_) [n:127.0.0.1:36941_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 297201 INFO  (coreLoadExecutor-704-thread-1-processing-n:127.0.0.1:36941_) [n:127.0.0.1:36941_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 297201 INFO  (OverseerStateUpdate-94628640064339974-127.0.0.1:51067_-n_0000000000) [n:127.0.0.1:51067_    ] o.a.s.c.Overseer processMessage: que

[...truncated too long message...]

junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4449764 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[5DE68C2A41FF7D9B]-worker) [n:127.0.0.1:39157_ c:addReplicaColl s:shard1 r:core_node4 x:addReplicaColl_shard1_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Oct 03, 2015 9:36:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {a_i1=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), intDefault=PostingsFormat(name=Direct), a_t=PostingsFormat(name=Direct), text=FST50, range_facet_l_dv=PostingsFormat(name=MockRandom), other_tl1=PostingsFormat(name=Direct), id=PostingsFormat(name=MockRandom), timestamp=PostingsFormat(name=Direct), range_facet_i_dv=PostingsFormat(name=Direct), _version_=PostingsFormat(name=Direct)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_PY, timezone=America/Moncton
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=306984680,total=532676608
   [junit4]   2> NOTE: All tests run in this JVM: [TestQuerySenderListener, AtomicUpdatesTest, TestCursorMarkWithoutUniqueKey, SpellCheckComponentTest, TestQueryUtils, DocumentBuilderTest, TestMissingGroups, SpellCheckCollatorTest, TestEmbeddedSolrServerConstructors, PKIAuthenticationIntegrationTest, BigEndianAscendingWordSerializerTest, CachingDirectoryFactoryTest, SolrPluginUtilsTest, TestDFRSimilarityFactory, OutOfBoxZkACLAndCredentialsProvidersTest, HLLUtilTest, DistributedQueueTest, SolrCloudExampleTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestFieldCollectionResource, TestSortByMinMaxFunction, TestPartialUpdateDeduplication, SuggesterFSTTest, AnalyticsMergeStrategyTest, MinimalSchemaTest, WordBreakSolrSpellCheckerTest, TestCloudManagedSchema, TestConfigSetsAPIZkFailure, TestDistributedGrouping, TestCodecSupport, TestWordDelimiterFilterFactory, TestFiltering, TestFieldTypeCollectionResource, OverriddenZkACLAndCredentialsProvidersTest, TestSchemaVersionResource, IgnoreCommitOptimizeUpdateProcessorFactoryTest, CollectionsAPIDistributedZkTest, TestUniqueKeyFieldResource, TermsComponentTest, SolrInfoMBeanTest, TestConfigOverlay, NumericFieldsTest, TestQueryTypes, TestCoreContainer, TestStressVersions, ZkStateReaderTest, CoreAdminRequestStatusTest, ZkSolrClientTest, TestOverriddenPrefixQueryForCustomFieldType, SortByFunctionTest, TestManagedResourceStorage, TestBinaryResponseWriter, DistributedFacetPivotSmallAdvancedTest, MoreLikeThisHandlerTest, DeleteLastCustomShardedReplicaTest, TestRestManager, TestSuggestSpellingConverter, TestBM25SimilarityFactory, TestUtils, TestTrie, HdfsRecoveryZkTest, SaslZkACLProviderTest, TestCloudManagedSchemaConcurrent, DOMUtilTest, TestLMJelinekMercerSimilarityFactory, ConvertedLegacyTest, TestHashPartitioner, SSLMigrationTest, ChaosMonkeySafeLeaderTest, TestHashQParserPlugin, TestRTimerTree, SimpleMLTQParserTest, TestRuleBasedAuthorizationPlugin, HdfsSyncSliceTest, CollectionStateFormat2Test, RankQueryTest, TestCloudSchemaless, SimplePostToolTest, UpdateRequestProcessorFactoryTest, TestSolrCoreProperties, DistributedTermsComponentTest, DateRangeFieldTest, SpellPossibilityIteratorTest, ZkControllerTest, ShardRoutingCustomTest, DistanceFunctionTest, TestReloadDeadlock, BasicDistributedZkTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, LeaderElectionTest, TestRandomDVFaceting, ZkCLITest, AutoCommitTest, TestRecovery, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestStressReorder, HardAutoCommitTest, TestRangeQuery, SimpleFacetsTest, SolrCmdDistributorTest, BadIndexSchemaTest, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, HighlighterTest, SoftAutoCommitTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SolrRequestParserTest, DocValuesTest, SuggesterTest, SpatialFilterTest, SuggesterWFSTTest, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, RequiredFieldsTest, TestSolrQueryParser, TestArbitraryIndexDir, FastVectorHighlighterTest, LoggingHandlerTest, RegexBoostProcessorTest, TestCollationField, MBeansHandlerTest, QueryParsingTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, BinaryUpdateRequestHandlerTest, PingRequestHandlerTest, CSVRequestHandlerTest, TestLFUCache, PreAnalyzedFieldTest, DateFieldTest, TestLRUCache, SliceStateTest, SystemInfoHandlerTest, FileUtilsTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, SolrTestCaseJ4Test, TestDistributedMissingSort, TestTolerantSearch, ActionThrottleTest, AssignTest, AsyncMigrateRouteKeyTest, ConnectionManagerTest, DeleteReplicaTest, DeleteShardTest, DistribDocExpirationUpdateProcessorTest, MultiThreadedOCPTest, OverseerStatusTest, OverseerTaskQueueTest, RecoveryAfterSoftCommitTest, SolrXmlInZkTest, TestCollectionAPI, TestDistribDocBasedVersion, TestLeaderInitiatedRecoveryThread, TestRandomRequestDistribution, TestRebalanceLeaders, TestRequestStatusCollectionAPI, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [492/539] on J1 in 502.66s, 1 test, 1 error <<< FAILURES!

[...truncated 155 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 2 errors, 29 ignored (17 assumptions) [seed: 5DE68C2A41FF7D9B]

Total time: 195 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 974 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/974/

6 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=13921, name=qtp431643373-13921-acceptor-0@47154c68-ServerConnector@71abfd13{HTTP/1.1}{127.0.0.1:55633}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=13939, name=Thread-5873, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1155)    3) Thread[id=14923, name=searcherExecutor-4851-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=13965, name=searcherExecutor-4221-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=14000, name=searcherExecutor-4238-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=13928, name=org.eclipse.jetty.server.session.HashSessionManager@605cd380Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=14811, name=searcherExecutor-4758-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=14735, name=searcherExecutor-4658-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=15187, name=zkCallback-2053-thread-11-processing-n:127.0.0.1:55633_x, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:200)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:138)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=15083, name=searcherExecutor-4926-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=13920, name=qtp431643373-13920-selector-ServerConnectorManager@527424b2/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=13927, name=Scheduler-1023735231, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=14620, name=qtp431643373-14620, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=15032, name=searcherExecutor-4941-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=15364, name=zkCallback-2053-thread-11-processing-n:127.0.0.1:55633_x-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   16) Thread[id=14371, name=searcherExecutor-4513-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=13943, name=searcherExecutor-4208-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=13956, name=Scheduler-430410445, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=14092, name=searcherExecutor-4330-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=14143, name=searcherExecutor-4375-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=14588, name=qtp431643373-14588, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=13919, name=qtp431643373-13919-selector-ServerConnectorManager@527424b2/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=14971, name=searcherExecutor-4872-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=14800, name=searcherExecutor-4732-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=16427, name=qtp431643373-16427, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=14082, name=searcherExecutor-4314-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=15074, name=searcherExecutor-4912-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=15066, name=searcherExecutor-4923-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=14012, name=searcherExecutor-4254-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=14278, name=qtp431643373-14278, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=14083, name=searcherExecutor-4319-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=14148, name=searcherExecutor-4380-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=14173, name=searcherExecutor-4415-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=14741, name=searcherExecutor-4653-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=14074, name=qtp431643373-14074, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=15361, name=zkCallback-2053-thread-11-processing-n:127.0.0.1:55633_x-SendThread(127.0.0.1:58806), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=13921, name=qtp431643373-13921-acceptor-0@47154c68-ServerConnector@71abfd13{HTTP/1.1}{127.0.0.1:55633}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=13939, name=Thread-5873, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:503)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1155)
   3) Thread[id=14923, name=searcherExecutor-4851-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=13965, name=searcherExecutor-4221-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=14000, name=searcherExecutor-4238-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=13928, name=org.eclipse.jetty.server.session.HashSessionManager@605cd380Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=14811, name=searcherExecutor-4758-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=14735, name=searcherExecutor-4658-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=15187, name=zkCallback-2053-thread-11-processing-n:127.0.0.1:55633_x, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:200)
        at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:138)
        at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)
        at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)
        at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=15083, name=searcherExecutor-4926-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=13920, name=qtp431643373-13920-selector-ServerConnectorManager@527424b2/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
        at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=13927, name=Scheduler-1023735231, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=14620, name=qtp431643373-14620, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=15032, name=searcherExecutor-4941-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=15364, name=zkCallback-2053-thread-11-processing-n:127.0.0.1:55633_x-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  16) Thread[id=14371, name=searcherExecutor-4513-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  17) Thread[id=13943, name=searcherExecutor-4208-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  18) Thread[id=13956, name=Scheduler-430410445, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  19) Thread[id=14092, name=searcherExecutor-4330-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  20) Thread[id=14143, name=searcherExecutor-4375-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  21) Thread[id=14588, name=qtp431643373-14588, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  22) Thread[id=13919, name=qtp431643373-13919-selector-ServerConnectorManager@527424b2/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
        at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
  23) Thread[id=14971, name=searcherExecutor-4872-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  24) Thread[id=14800, name=searcherExecutor-4732-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  25) Thread[id=16427, name=qtp431643373-16427, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  26) Thread[id=14082, name=searcherExecutor-4314-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  27) Thread[id=15074, name=searcherExecutor-4912-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  28) Thread[id=15066, name=searcherExecutor-4923-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  29) Thread[id=14012, name=searcherExecutor-4254-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  30) Thread[id=14278, name=qtp431643373-14278, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  31) Thread[id=14083, name=searcherExecutor-4319-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  32) Thread[id=14148, name=searcherExecutor-4380-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  33) Thread[id=14173, name=searcherExecutor-4415-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  34) Thread[id=14741, name=searcherExecutor-4653-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  35) Thread[id=14074, name=qtp431643373-14074, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  36) Thread[id=15361, name=zkCallback-2053-thread-11-processing-n:127.0.0.1:55633_x-SendThread(127.0.0.1:58806), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
	at __randomizedtesting.SeedInfo.seed([3942EFAC8F5471FF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=15074, name=searcherExecutor-4912-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=14735, name=searcherExecutor-4658-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=14800, name=searcherExecutor-4732-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=13928, name=org.eclipse.jetty.server.session.HashSessionManager@605cd380Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=14082, name=searcherExecutor-4314-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=14811, name=searcherExecutor-4758-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=13965, name=searcherExecutor-4221-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=14000, name=searcherExecutor-4238-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=14923, name=searcherExecutor-4851-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=13919, name=qtp431643373-13919-selector-ServerConnectorManager@527424b2/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=14971, name=searcherExecutor-4872-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=14741, name=searcherExecutor-4653-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=14092, name=searcherExecutor-4330-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=14143, name=searcherExecutor-4375-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=14173, name=searcherExecutor-4415-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=14371, name=searcherExecutor-4513-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=13956, name=Scheduler-430410445, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=13943, name=searcherExecutor-4208-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=15032, name=searcherExecutor-4941-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=14148, name=searcherExecutor-4380-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=13920, name=qtp431643373-13920-selector-ServerConnectorManager@527424b2/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=13927, name=Scheduler-1023735231, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=15187, name=zkCallback-2053-thread-11, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=14012, name=searcherExecutor-4254-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=15066, name=searcherExecutor-4923-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=14083, name=searcherExecutor-4319-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=15083, name=searcherExecutor-4926-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=15074, name=searcherExecutor-4912-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=14735, name=searcherExecutor-4658-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=14800, name=searcherExecutor-4732-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=13928, name=org.eclipse.jetty.server.session.HashSessionManager@605cd380Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=14082, name=searcherExecutor-4314-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=14811, name=searcherExecutor-4758-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=13965, name=searcherExecutor-4221-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=14000, name=searcherExecutor-4238-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=14923, name=searcherExecutor-4851-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=13919, name=qtp431643373-13919-selector-ServerConnectorManager@527424b2/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.nio.ch.IOUtil.drain(Native Method)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
        at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=14971, name=searcherExecutor-4872-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=14741, name=searcherExecutor-4653-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=14092, name=searcherExecutor-4330-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=14143, name=searcherExecutor-4375-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Met

[...truncated too long message...]

0) Thread[id=14148, name=searcherExecutor-4380-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   21) Thread[id=13920, name=qtp431643373-13920-selector-ServerConnectorManager@527424b2/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
   [junit4]    >         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
   [junit4]    >         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
   [junit4]    >         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   22) Thread[id=13927, name=Scheduler-1023735231, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   23) Thread[id=15187, name=zkCallback-2053-thread-11, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   24) Thread[id=14012, name=searcherExecutor-4254-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   25) Thread[id=15066, name=searcherExecutor-4923-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   26) Thread[id=14083, name=searcherExecutor-4319-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   27) Thread[id=15083, name=searcherExecutor-4926-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3942EFAC8F5471FF]:0)
   [junit4] Completed [530/539] on J1 in 3057.01s, 1 test, 3 errors <<< FAILURES!

[...truncated 28 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151003_032610_587.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid13567.hprof ...
   [junit4] Heap dump file created [686772697 bytes in 9.916 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 14 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 2 suite-level errors, 3 errors, 1 failure, 87 ignored (17 assumptions) [seed: 3942EFAC8F5471FF]

Total time: 214 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 973 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/973/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:34287: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34287: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=30692, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=30692, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35798: collection already exists: awholynewstresscollection_collection3_3
	at __randomizedtesting.SeedInfo.seed([C9BB8C2B9752CFE4]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 11581 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/init-core-data-001
   [junit4]   2> 4349994 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 4349994 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4350083 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4350106 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4350120 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4350162 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_54538_hdfs____5x28lh/webapp
   [junit4]   2> 4350302 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4350960 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54538
   [junit4]   2> 4351243 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4351246 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4351275 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_56509_datanode____.56hk2k/webapp
   [junit4]   2> 4351548 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4351906 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56509
   [junit4]   2> 4352191 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4352193 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4352222 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54465_datanode____r1qsij/webapp
   [junit4]   2> 4352244 INFO  (IPC Server handler 8 on 40525) [    ] BlockStateChange BLOCK* processReport: from storage DS-a197d61d-8460-429c-9839-67df572b7306 node DatanodeRegistration(127.0.0.1, datanodeUuid=9ff7864e-0265-4ad3-bb27-8d580fbeacf8, infoPort=56509, ipcPort=47662, storageInfo=lv=-56;cid=testClusterID;nsid=266012631;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4352244 INFO  (IPC Server handler 8 on 40525) [    ] BlockStateChange BLOCK* processReport: from storage DS-6fe562b5-2f05-4a35-adee-176e93ad0d96 node DatanodeRegistration(127.0.0.1, datanodeUuid=9ff7864e-0265-4ad3-bb27-8d580fbeacf8, infoPort=56509, ipcPort=47662, storageInfo=lv=-56;cid=testClusterID;nsid=266012631;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4352347 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4352825 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C9BB8C2B9752CFE4]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54465
   [junit4]   2> 4352933 INFO  (IPC Server handler 3 on 40525) [    ] BlockStateChange BLOCK* processReport: from storage DS-6ac1c38b-fd37-4e0d-8b0f-e7d581e148a6 node DatanodeRegistration(127.0.0.1, datanodeUuid=be36c3a4-ca63-404e-be80-7d6291a842f5, infoPort=54465, ipcPort=40258, storageInfo=lv=-56;cid=testClusterID;nsid=266012631;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4352934 INFO  (IPC Server handler 3 on 40525) [    ] BlockStateChange BLOCK* processReport: from storage DS-619b496c-da47-48e1-bcf1-dd912fb7c94a node DatanodeRegistration(127.0.0.1, datanodeUuid=be36c3a4-ca63-404e-be80-7d6291a842f5, infoPort=54465, ipcPort=40258, storageInfo=lv=-56;cid=testClusterID;nsid=266012631;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4353014 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4353015 INFO  (Thread-14965) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4353015 INFO  (Thread-14965) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4353115 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.ZkTestServer start zk server on port:47617
   [junit4]   2> 4353115 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4353127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4353135 INFO  (zkCallback-3038-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56f81590 name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4353135 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4353136 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4353136 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4353139 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4353141 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4353142 INFO  (zkCallback-3039-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21f586b3 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4353142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4353142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4353142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 4353145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 4353146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 4353148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 4353149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4353149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4353152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4353152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 4353154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4353154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4353156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4353156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4353158 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4353158 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4353159 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4353160 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 4353161 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4353161 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4353181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4353181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4353183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4353183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4353184 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4353185 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4353186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4353186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4353188 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4353216 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4353228 INFO  (zkCallback-3040-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@665f304f name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4353228 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4353229 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4353229 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4353230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4353248 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4353257 INFO  (zkCallback-3041-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@45383d34 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4353257 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4353257 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4353258 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 4353258 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 4353260 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 4353260 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 4353262 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 4353262 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 4353263 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4353264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4353265 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 4353265 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 4353266 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 4353267 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 4353268 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 4353268 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 4353269 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 4353270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 4353271 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4353271 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4353272 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 4353273 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 4353274 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 4353274 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 4353275 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 4353275 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 4353718 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores/collection1
   [junit4]   2> 4353719 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4353736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@28d69891{/,null,AVAILABLE}
   [junit4]   2> 4353737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@20ba6c89{HTTP/1.1}{127.0.0.1:48887}
   [junit4]   2> 4353737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.Server Started @4357370ms
   [junit4]   2> 4353737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:40525/hdfs__localhost_40525__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001_tempDir-002_control_data, hostContext=/, hostPort=48887, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores}
   [junit4]   2> 4353737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 4353737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/'
   [junit4]   2> 4353757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4353776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4353781 INFO  (zkCallback-3042-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@cae3315 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4353781 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4353782 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4353792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4353792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/solr.xml
   [junit4]   2> 4353807 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores
   [junit4]   2> 4353807 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoreContainer New CoreContainer 949093131
   [junit4]   2> 4353807 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/]
   [junit4]   2> 4353808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/lib
   [junit4]   2> 4353808 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/lib).
   [junit4]   2> 4353816 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4353819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4353819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4353819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4353819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47617/solr
   [junit4]   2> 4353819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4353820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4353822 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4353830 INFO  (zkCallback-3044-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8c27510 name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4353832 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4353832 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4353841 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4353847 INFO  (zkCallback-3045-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38dd8ee7 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4353848 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4353849 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 4353850 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 4353853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 4353854 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 4353856 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 4353857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 4353858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 4353860 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 4353861 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 4353862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48887_
   [junit4]   2> 4353862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48887_
   [junit4]   2> 4353863 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 4353864 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 4353865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4353866 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94619421874323462-127.0.0.1:48887_-n_0000000000
   [junit4]   2> 4353866 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48887_
   [junit4]   2> 4353866 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 4353867 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer Overseer (id=94619421874323462-127.0.0.1:48887_-n_0000000000) starting
   [junit4]   2> 4353869 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 4353873 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4353881 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 4353882 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4353882 INFO  (OverseerCollectionConfigSetProcessor-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 4353894 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4353913 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4353913 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4353914 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores
   [junit4]   2> 4353915 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=control_collection, config=solrconfig.xml, coreNodeName=, shard=, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores/collection1, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores/collection1/}
   [junit4]   2> 4353915 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores/collection1/
   [junit4]   2> 4353915 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4353916 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 4353916 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [n:127.0.0.1:48887_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4353917 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4353917 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4353919 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:48887",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:48887_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 4353919 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:48887",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:48887_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 4353920 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4353920 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4353920 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4353922 INFO  (zkCallback-3045-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4353922 INFO  (zkCallback-3045-thread-2-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4354920 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4354921 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 4354921 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4354921 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4354922 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4354922 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores/collection1/'
   [junit4]   2> 4354949 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4354956 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4354995 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 4355066 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4355067 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 4355073 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4355374 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4355387 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 4355389 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 4355410 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4355413 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4355423 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4355424 WARN  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4355424 WARN  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4355424 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4355425 WARN  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4355425 WARN  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4355426 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 4355426 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4355426 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 4355427 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:40525/solr_hdfs_home
   [junit4]   2> 4355427 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4355427 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 4355427 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b51c44
   [junit4]   2> 4355427 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40525/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4355522 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:40525/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4355523 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:40525/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 4355548 WARN  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:40525/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4355548 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40525/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4355588 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:40525/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4355588 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=78.7978515625, floorSegmentMB=1.955078125, forceMergeDeletesPctAllowed=1.6096643303880698, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.45784176023457757
   [junit4]   2> 4355701 INFO  (IPC Server handler 3 on 40525) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35081 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6fe562b5-2f05-4a35-adee-176e93ad0d96:NORMAL:127.0.0.1:40041|RBW], ReplicaUnderConstruction[[DISK]DS-6ac1c38b-fd37-4e0d-8b0f-e7d581e148a6:NORMAL:127.0.0.1:35081|RBW]]} size 0
   [junit4]   2> 4355703 INFO  (IPC Server handler 4 on 40525) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40041 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ac1c38b-fd37-4e0d-8b0f-e7d581e148a6:NORMAL:127.0.0.1:35081|RBW], ReplicaUnderConstruction[[DISK]DS-a197d61d-8460-429c-9839-67df572b7306:NORMAL:127.0.0.1:40041|FINALIZED]]} size 0
   [junit4]   2> 4355707 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6de80af lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f3545d7; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4355707 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4355709 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:40525/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 4355713 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4355713 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4355714 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4355714 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4355714 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4355714 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4355714 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4355714 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4355715 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4355717 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4355719 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4355720 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4355721 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4355727 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/properties,/admin/system,/admin/mbeans,/update/json,/admin/ping,/update/csv,/config,/admin/luke,/admin/logging,/update,/update/json/docs,/get,/admin/segments,/admin/file,/schema,/admin/threads,/admin/plugins,standard,/replication
   [junit4]   2> 4355728 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 4355729 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4355729 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 4355742 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 4355742 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 4355743 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=12, maxMergedSegmentMB=34.984375, floorSegmentMB=1.65625, forceMergeDeletesPctAllowed=23.733475683926926, segmentsPerTier=43.0, maxCFSSegmentSizeMB=0.2333984375, noCFSRatio=0.8666214776254545
   [junit4]   2> 4355752 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6de80af lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f3545d7; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4355752 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4355752 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7bbb2f02[collection1] main
   [junit4]   2> 4355752 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4355753 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4355753 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4355754 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4355754 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 4355754 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4355755 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4355755 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4355755 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 4355755 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4355756 INFO  (searcherExecutor-7823-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7bbb2f02[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4355757 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 4355757 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 4355757 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 4355757 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513910752863322112
   [junit4]   2> 4355757 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1513910752863322112
   [junit4]   2> 4355757 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 4355758 INFO  (coreLoadExecutor-7822-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 4355758 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:48887 collection:control_collection shard:shard1
   [junit4]   2> 4355759 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 4355762 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94619421874323462-core_node1-n_0000000000
   [junit4]   2> 4355762 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 4355764 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4355764 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 4355764 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4355764 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48887/collection1/
   [junit4]   2> 4355765 INFO  (zkCallback-3045-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4355766 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4355766 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:48887/collection1/ has no replicas
   [junit4]   2> 4355766 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48887/collection1/ shard1
   [junit4]   2> 4355765 INFO  (zkCallback-3045-thread-2-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4355766 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 4355769 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 4355771 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:48887",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 4355874 INFO  (zkCallback-3045-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4355874 INFO  (zkCallback-3045-thread-2-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4355882 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:48887/collection1/ and leader is http://127.0.0.1:48887/collection1/
   [junit4]   2> 4355882 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48887
   [junit4]   2> 4355882 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4355882 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 4355882 INFO  (coreZkRegister-7816-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:48887_ c:control_collection x:collection1) [n:127.0.0.1:48887_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4355885 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:48887",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:48887_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 4355885 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:48887",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:48887_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 4355919 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4355941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4355957 INFO  (zkCallback-3047-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d7df201 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4355957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4355957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4355958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4355960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 4355960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 4355960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4355969 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4355973 INFO  (zkCallback-3048-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79a31aad name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4355973 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4355973 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4355975 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 4355975 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 4355976 INFO  (zkCallback-3045-thread-1-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4355976 INFO  (zkCallback-3045-thread-2-processing-n:127.0.0.1:48887_) [n:127.0.0.1:48887_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4355980 INFO  (zkCallback-3047-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4356079 INFO  (OverseerStateUpdate-94619421874323462-127.0.0.1:48887_-n_0000000000) [n:127.0.0.1:48887_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 4356349 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001/cores/collection1
   [junit4]   2> 4356350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001
   [junit4]   2> 4356351 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4356424 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@658c0e16{/,null,AVAILABLE}
   [junit4]   2> 4356425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7c9b563{HTTP/1.1}{127.0.0.1:53646}
   [junit4]   2> 4356425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.e.j.s.Server Started @4360058ms
   [junit4]   2> 4356425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, hostPort=53646, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:40525/hdfs__localhost_40525__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001_tempDir-002_jetty1}
   [junit4]   2> 4356425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 4356425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001/'
   [junit4]   2> 4356443 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4356447 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4356448 INFO  (zkCallback-3049-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27927fc4 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4356449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4356449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4356450 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4356450 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001/solr.xml
   [junit4]   2> 4356465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001/cores
   [junit4]   2> 4356465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoreContainer New CoreContainer 2031373959
   [junit4]   2> 4356465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001/]
   [junit4]   2> 4356465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C9BB8C2B9752CFE4-001/shard-1-001/lib
   [junit4]   2> 4356465 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[C9BB8C2B9752CFE4]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build

[...truncated too long message...]

ted from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@577c5693[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1901]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 7168850 ERROR (zkCallback-2178-thread-23-processing-n:127.0.0.1:45341_-EventThread) [n:127.0.0.1:45341_ c:acollectionafterbaddelete s:shard1 r:core_node2 x:acollectionafterbaddelete_shard1_replica1] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@7523f000 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@577c5693[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1901]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 7168851 ERROR (zkCallback-2178-thread-23-processing-n:127.0.0.1:45341_-EventThread) [n:127.0.0.1:45341_ c:acollectionafterbaddelete s:shard1 r:core_node2 x:acollectionafterbaddelete_shard1_replica1] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@1a9d7c06 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@577c5693[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1901]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 7168851 ERROR (zkCallback-2178-thread-23-processing-n:127.0.0.1:45341_-EventThread) [n:127.0.0.1:45341_ c:acollectionafterbaddelete s:shard1 r:core_node2 x:acollectionafterbaddelete_shard1_replica1] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@4cdf58b3 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@577c5693[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1901]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval), other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), text=FSTOrd50, range_facet_i_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_i1=PostingsFormat(name=LuceneVarGapDocFreqInterval), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Asserting)}, sim=DefaultSimilarity, locale=el_GR, timezone=Pacific/Guam
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=54616072,total=471334912
   [junit4]   2> NOTE: All tests run in this JVM: [AutoCommitTest, TestCollationField, TestStressReorder, TestWriterPerf, TestDistribIDF, HdfsDirectoryFactoryTest, TestSolrQueryParser, LukeRequestHandlerTest, MergeStrategyTest, SuggesterWFSTTest, TestCharFilters, HdfsUnloadDistributedZkTest, AddBlockUpdateTest, TestSolrConfigHandlerCloud, TestFastWriter, TestLeaderInitiatedRecoveryThread, DirectSolrConnectionTest, TestJsonFacets, SmileWriterTest, TestReplicationHandlerBackup, TestCollapseQParserPlugin, TestSort, UUIDFieldTest, TestExactSharedStatsCache, RulesTest, TestFieldTypeResource, HdfsDirectoryTest, ResponseHeaderTest, TestConfigSetsAPIExclusivity, TestDefaultSimilarityFactory, FieldMutatingUpdateProcessorTest, TestShortCircuitedRequests, TestDistributedMissingSort, SolrTestCaseJ4Test, DistributedFacetPivotLongTailTest, HdfsBasicDistributedZkTest, TestLeaderElectionZkExpiry, TestAddFieldRealTimeGet, ShowFileRequestHandlerTest, OverseerTest, TermVectorComponentDistributedTest, TestDefaultSearchFieldResource, TestXIncludeConfig, TestExtendedDismaxParser, SpatialRPTFieldTypeTest, TestPerFieldSimilarity, HardAutoCommitTest, TestSolrCloudWithKerberosAlt, TestDefaultStatsCache, TestSolrDynamicMBean, TestMinMaxOnMultiValuedField, TestSolrQueryParserResource, ShardSplitTest, TestSearchPerf, TestSolrIndexConfig, HdfsWriteToMultipleCollectionsTest, TestPostingsSolrHighlighter, SearchHandlerTest, ChangedSchemaMergeTest, TestReversedWildcardFilterFactory, ClusterStateTest, BitVectorTest, DirectUpdateHandlerOptimizeTest, TestJettySolrRunner, PreAnalyzedUpdateProcessorTest, TestSystemIdResolver, ZkNodePropsTest, TestElisionMultitermQuery, TestNoOpRegenerator, TestDynamicLoading, RequestLoggingTest, BJQParserTest, DistributedExpandComponentTest, BadComponentTest, SecurityConfHandlerTest, TestAuthorizationFramework, TestReload, QueryResultKeyTest, TestSolrQueryParserDefaultOperatorResource, TestRemoveLastDynamicCopyField, DirectUpdateHandlerTest, FieldAnalysisRequestHandlerTest, PluginInfoTest, TestJmxIntegration, AsyncCallRequestStatusResponseTest, TestLRUCache, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [539/539] on J2 in 4899.26s, 1 test, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 2 errors, 29 ignored (17 assumptions) [seed: C9BB8C2B9752CFE4]

Total time: 244 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 972 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/972/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=11987, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11987, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51105: collection already exists: awholynewstresscollection_collection2_3
	at __randomizedtesting.SeedInfo.seed([D47E75BC6CC1D0F5]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=26614, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=26614, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50601/u_jv: Could not find collection : awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([D47E75BC6CC1D0F5]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10745 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/init-core-data-001
   [junit4]   2> 1350240 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D47E75BC6CC1D0F5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1350240 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D47E75BC6CC1D0F5]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1350247 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1350248 INFO  (Thread-6320) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1350248 INFO  (Thread-6320) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1350348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.ZkTestServer start zk server on port:44727
   [junit4]   2> 1350348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1350348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1350351 INFO  (zkCallback-1591-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14495124 name:ZooKeeperConnection Watcher:127.0.0.1:44727 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1350352 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1350352 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1350352 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1350355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1350356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1350359 INFO  (zkCallback-1592-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d06f906 name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1350359 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1350360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1350360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1350363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1350365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1350368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1350371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1350371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1350377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1350378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1350384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1350384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1350386 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1350386 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1350391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1350392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1350395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1350396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1350398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1350399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1350401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1350401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1350405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1350405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1350408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1350408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1350412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1350412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1350415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1350416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1350417 INFO  (zkCallback-1593-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b19a85e name:ZooKeeperConnection Watcher:127.0.0.1:44727 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1350417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1350418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1350418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1350422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1350423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1350424 INFO  (zkCallback-1594-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4f03ceff name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1350425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1350426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1350427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1350427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1350430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1350431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1350433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1350433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1350435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1350436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1350439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1350439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1350442 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1350443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1350445 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1350445 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1350447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1350447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1350449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1350449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1350451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1350452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1350455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1350455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1350458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1350458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1350854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1
   [junit4]   2> 1350856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1350867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@374f5157{/,null,AVAILABLE}
   [junit4]   2> 1350868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@19c526{HTTP/1.1}{127.0.0.1:51105}
   [junit4]   2> 1350868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.Server Started @1354948ms
   [junit4]   2> 1350868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=51105, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/tempDir-001/control/data}
   [junit4]   2> 1350868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1350868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/'
   [junit4]   2> 1350892 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1350893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1350895 INFO  (zkCallback-1595-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c3ca717 name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1350895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1350895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1350897 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1350897 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/solr.xml
   [junit4]   2> 1350915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores
   [junit4]   2> 1350915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoreContainer New CoreContainer 1017721489
   [junit4]   2> 1350915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/]
   [junit4]   2> 1350915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/lib
   [junit4]   2> 1350915 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/lib).
   [junit4]   2> 1350930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1350932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1350932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1350932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1350932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44727/solr
   [junit4]   2> 1350932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1350933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1350935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1350936 INFO  (zkCallback-1597-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57b2411d name:ZooKeeperConnection Watcher:127.0.0.1:44727 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1350938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1350939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1350943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1350944 INFO  (zkCallback-1598-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@585571af name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1350944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1350946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1350951 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1350961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1350964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1350965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1350968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1350970 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1350975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1350981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1350983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51105_
   [junit4]   2> 1350983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51105_
   [junit4]   2> 1350985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1350987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1350990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1350992 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94611214947123206-127.0.0.1:51105_-n_0000000000
   [junit4]   2> 1350993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51105_
   [junit4]   2> 1350993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1350998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer Overseer (id=94611214947123206-127.0.0.1:51105_-n_0000000000) starting
   [junit4]   2> 1351002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1351026 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1351027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1351030 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1351036 INFO  (OverseerCollectionConfigSetProcessor-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1351056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1351056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1351059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores
   [junit4]   2> 1351060 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1, transient=false, loadOnStartup=true, config=solrconfig.xml, collection=control_collection, shard=, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/, dataDir=data/, name=collection1}
   [junit4]   2> 1351060 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/
   [junit4]   2> 1351060 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1351078 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1351078 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1351081 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1351087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1351087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:51105_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1351087 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51105_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51105"} current state version: 0
   [junit4]   2> 1351088 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51105_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51105"}
   [junit4]   2> 1351088 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1351090 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1351092 INFO  (zkCallback-1598-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1352082 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1352082 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1352083 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1352083 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1352085 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1352085 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/'
   [junit4]   2> 1352108 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1352118 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1352135 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1352198 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1352201 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1352213 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1352597 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1352607 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1352609 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1352623 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1352627 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1352633 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1352634 WARN  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1352634 WARN  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1352634 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1352636 WARN  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1352636 WARN  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1352638 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1352639 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1352639 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1352639 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1352639 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a941c4e
   [junit4]   2> 1352640 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/data
   [junit4]   2> 1352640 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1352640 WARN  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1352641 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/data/index
   [junit4]   2> 1352641 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1860541805, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1352642 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7b8292dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3977e8dd),segFN=segments_1,generation=1}
   [junit4]   2> 1352642 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1352648 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1352648 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1352649 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1352649 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1352649 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1352649 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1352665 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1352665 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1352665 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1352668 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1352664 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/data/
   [junit4]   2> 1352669 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1352673 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1352675 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1352676 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1352686 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,/admin/segments,/admin/properties,/replication,/admin/ping,/update/json,/admin/logging,/get,/admin/luke,/admin/system,standard,/admin/threads,/schema,/update,/update/csv,/admin/file,/admin/plugins,/admin/mbeans,/config
   [junit4]   2> 1352690 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1352693 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1352693 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1352694 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1352694 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1352695 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=16.2978515625, floorSegmentMB=0.283203125, forceMergeDeletesPctAllowed=11.864999197586176, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1352695 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7b8292dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3977e8dd),segFN=segments_1,generation=1}
   [junit4]   2> 1352695 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1352696 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4ad56575[collection1] main
   [junit4]   2> 1352696 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1352698 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1352698 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1352698 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1352698 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1352698 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1352699 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1352699 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1352699 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1352699 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1352700 INFO  (searcherExecutor-2522-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4ad56575[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1352700 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1352700 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1352700 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1352700 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513779441722982400
   [junit4]   2> 1352702 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1513779441722982400
   [junit4]   2> 1352703 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1352703 INFO  (coreLoadExecutor-2521-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1352704 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:51105 collection:control_collection shard:shard1
   [junit4]   2> 1352705 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1352709 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94611214947123206-core_node1-n_0000000000
   [junit4]   2> 1352710 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1352713 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1352713 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1352713 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51105/collection1/
   [junit4]   2> 1352713 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1352713 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51105/collection1/ has no replicas
   [junit4]   2> 1352713 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1352714 INFO  (zkCallback-1598-thread-1-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1352713 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51105/collection1/ shard1
   [junit4]   2> 1352718 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1352720 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1352724 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51105",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1352829 INFO  (zkCallback-1598-thread-2-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1352878 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:51105/collection1/ and leader is http://127.0.0.1:51105/collection1/
   [junit4]   2> 1352878 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51105
   [junit4]   2> 1352878 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1352878 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1352878 INFO  (coreZkRegister-2515-thread-1-processing-n:127.0.0.1:51105_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:51105_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1352881 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:51105_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51105"} current state version: 3
   [junit4]   2> 1352882 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:51105_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:51105"}
   [junit4]   2> 1352986 INFO  (zkCallback-1598-thread-2-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1353089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1353089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1353095 INFO  (zkCallback-1600-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59c583ac name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1353096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1353096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1353096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1353102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1353102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1353102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1353103 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1353114 INFO  (zkCallback-1601-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@373027ed name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1353114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1353115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1353123 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 1353123 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1353228 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1353489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/cores/collection1
   [junit4]   2> 1353489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001
   [junit4]   2> 1353491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1353492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@83ff82e{/,null,AVAILABLE}
   [junit4]   2> 1353493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@19fcdd13{HTTP/1.1}{127.0.0.1:45675}
   [junit4]   2> 1353493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.e.j.s.Server Started @1357573ms
   [junit4]   2> 1353493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/cores, hostContext=/, solrconfig=solrconfig.xml, hostPort=45675, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/tempDir-001/jetty1}
   [junit4]   2> 1353493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1353493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/'
   [junit4]   2> 1353510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1353510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1353513 INFO  (zkCallback-1602-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a03e88d name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1353513 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1353513 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1353519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1353520 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/solr.xml
   [junit4]   2> 1353543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/cores
   [junit4]   2> 1353543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoreContainer New CoreContainer 579542374
   [junit4]   2> 1353543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/]
   [junit4]   2> 1353543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/lib
   [junit4]   2> 1353543 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/lib).
   [junit4]   2> 1353557 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1353559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1353560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1353560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1353560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44727/solr
   [junit4]   2> 1353560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1353560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1353561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1353563 INFO  (zkCallback-1604-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@488f77 name:ZooKeeperConnection Watcher:127.0.0.1:44727 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1353566 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1353567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1353569 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1353571 INFO  (zkCallback-1605-thread-1-processing-n:127.0.0.1:45675_) [n:127.0.0.1:45675_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34fff63f name:ZooKeeperConnection Watcher:127.0.0.1:44727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1353571 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1353582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1354593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45675_
   [junit4]   2> 1354594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45675_
   [junit4]   2> 1354595 INFO  (zkCallback-1600-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1354595 INFO  (zkCallback-1598-thread-2-processing-n:127.0.0.1:51105_) [n:127.0.0.1:51105_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1354598 INFO  (zkCallback-1605-thread-1-processing-n:127.0.0.1:45675_) [n:127.0.0.1:45675_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1354600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1354603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94611214947123211-127.0.0.1:45675_-n_0000000001
   [junit4]   2> 1354604 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94611214947123206-127.0.0.1:51105_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1354628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1354628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1354629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/cores
   [junit4]   2> 1354629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/cores/collection1/, dataDir=data/, configSetProperties=configsetprops.json, name=collection1, schema=schema.xml, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/cores/collection1, collection=collection1, shard=, loadOnStartup=true}
   [junit4]   2> 1354630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D47E75BC6CC1D0F5-001/shard-1-001/cores/collection1/
   [junit4]   2> 1354630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1354631 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1354631 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D47E75BC6CC1D0F5]) [n:127.0.0.1:45675_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1354632 INFO  (coreLoadExecutor-2532-thread-1-processing-n:127.0.0.1:45675_) [n:127.0.0.1:45675_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1354632 INFO  (coreLoadExecutor-2532-thread-1-processing-n:127.0.0.1:45675_) [n:127.0.0.1:45675_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1354634 INFO  (coreLoadExecutor-2532-thread-1-processing-n:127.0.0.1:45675_) [n:127.0.0.1:45675_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1354634 INFO  (coreLoadExecutor-2532-thread-1-processing-n:127.0.0.1:45675_) [n:127.0.0.1:45675_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1354635 INFO  (OverseerStateUpdate-94611214947123206-127.0.0.1:51105_-n_0000000000) [n:127.0.0.1:51105_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:45675_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "b

[...truncated too long message...]

4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4322730 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D47E75BC6CC1D0F5]-worker) [n:127.0.0.1:50794_u_jv c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 30, 2015 7:25:44 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ar_AE, timezone=SystemV/MST7MDT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=148440400,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [TestGroupingSearch, TestBulkSchemaAPI, TestLeaderInitiatedRecoveryThread, OverseerTest, BitVectorTest, TestCollationField, TestDefaultStatsCache, TestSimpleTrackingShardHandler, TestReplicaProperties, CloudMLTQParserTest, PluginInfoTest, HdfsUnloadDistributedZkTest, FacetPivotSmallTest, SolrCoreCheckLockOnStartupTest, ChangedSchemaMergeTest, TestPHPSerializedResponseWriter, ResponseHeaderTest, TestJettySolrRunner, LukeRequestHandlerTest, TestStressRecovery, SampleTest, PreAnalyzedUpdateProcessorTest, TestJmxIntegration, TestReload, TestSearchPerf, TestIndexingPerformance, PrimitiveFieldTypeTest, TestShardHandlerFactory, CoreAdminHandlerTest, TestManagedSchemaFieldResource, DebugComponentTest, DirectUpdateHandlerTest, BJQParserTest, DocExpirationUpdateProcessorFactoryTest, FieldAnalysisRequestHandlerTest, CurrencyFieldXmlFileTest, TestCopyFieldCollectionResource, TestSort, RegexBoostProcessorTest, DistribJoinFromCollectionTest, TestClassNameShortening, TestSystemIdResolver, TestNonDefinedSimilarityFactory, SuggesterWFSTTest, ScriptEngineTest, TestStandardQParsers, TestCharFilters, TestRemoveLastDynamicCopyField, TermVectorComponentDistributedTest, SuggesterTest, TestDefaultSimilarityFactory, BasicAuthIntegrationTest, HdfsDirectoryFactoryTest, RAMDirectoryFactoryTest, TestJsonFacets, XmlUpdateRequestHandlerTest, TestStressReorder, TestSolrIndexConfig, TestMinMaxOnMultiValuedField, CollectionReloadTest, SmileWriterTest, DistributedExpandComponentTest, LeaderFailoverAfterPartitionTest, BadComponentTest, TestReRankQParserPlugin, TestXIncludeConfig, NoCacheHeaderTest, TestRecovery, HdfsDirectoryTest, QueryResultKeyTest, TestShortCircuitedRequests, FieldMutatingUpdateProcessorTest, TestFastWriter, TestSolrConfigHandlerCloud, MergeStrategyTest, SolrTestCaseJ4Test, TestExceedMaxTermLength, TestAddFieldRealTimeGet, RequestLoggingTest, AddBlockUpdateTest, StatsComponentTest, SuggesterTSTTest, TestDynamicLoading, TestJsonRequest, QueryParsingTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, AutoCommitTest, TestDistributedGrouping, HardAutoCommitTest, TestCoreContainer, TestSolr4Spatial, SpellCheckComponentTest, QueryElevationComponentTest, TestFiltering, HighlighterTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, TestExtendedDismaxParser, SpellCheckCollatorTest, WordBreakSolrSpellCheckerTest, TestAtomicUpdateErrorCases, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, DocValuesMultiTest, XsltUpdateRequestHandlerTest, DisMaxRequestHandlerTest, TestWriterPerf, FileBasedSpellCheckerTest, RequestHandlersTest, TermVectorComponentTest, TestSolrQueryParser, LoggingHandlerTest, SolrPluginUtilsTest, JSONWriterTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, TestQuerySenderListener, UpdateParamsTest, AlternateDirectoryTest, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, TestBinaryField, TestConfig, TestFuzzyAnalyzedSuggestions, DirectSolrConnectionTest, TestLuceneMatchVersion, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestIBSimilarityFactory, ResourceLoaderTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, TestHighlightDedupGrouping, CloudExitableDirectoryReaderTest, DeleteInactiveReplicaTest, DistribDocExpirationUpdateProcessorTest, HttpPartitionTest, ShardSplitTest, TestMiniSolrCloudClusterSSL, TestSolrCloudWithKerberosAlt, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [539/539] on J2 in 1119.99s, 1 test, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 2 errors, 29 ignored (17 assumptions) [seed: D47E75BC6CC1D0F5]

Total time: 195 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 971 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/971/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=1857, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1857, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50250: Could not find collection : awholynewstresscollection_collection5_3
	at __randomizedtesting.SeedInfo.seed([55BD0976D413947D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=5359, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5359, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53360: Could not find collection : awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([55BD0976D413947D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10143 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/init-core-data-001
   [junit4]   2> 121970 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[55BD0976D413947D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 121970 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[55BD0976D413947D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 121976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 121976 INFO  (Thread-1082) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 121976 INFO  (Thread-1082) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 122076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.ZkTestServer start zk server on port:53827
   [junit4]   2> 122076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122078 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122102 INFO  (zkCallback-81-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7459376 name:ZooKeeperConnection Watcher:127.0.0.1:53827 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 122116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122119 INFO  (zkCallback-82-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@288c7c7d name:ZooKeeperConnection Watcher:127.0.0.1:53827/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 122123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 122125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 122129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 122132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 122132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 122138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 122138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 122145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 122145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 122154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 122154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 122160 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 122160 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 122163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 122163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 122167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 122167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 122172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 122172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 122178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 122178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 122181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 122181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 122183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 122184 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 122588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1
   [junit4]   2> 122589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 122591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34aab9eb{/,null,AVAILABLE}
   [junit4]   2> 122593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6437c127{HTTP/1.1}{127.0.0.1:50250}
   [junit4]   2> 122593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.Server Started @125801ms
   [junit4]   2> 122593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=50250, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores}
   [junit4]   2> 122594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 122594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/'
   [junit4]   2> 122609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122612 INFO  (zkCallback-83-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@71466c69 name:ZooKeeperConnection Watcher:127.0.0.1:53827/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122614 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 122614 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/solr.xml
   [junit4]   2> 122629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores
   [junit4]   2> 122629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1134912663
   [junit4]   2> 122629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/]
   [junit4]   2> 122629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/lib
   [junit4]   2> 122629 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/lib).
   [junit4]   2> 122637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 122638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 122638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 122639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 122639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53827/solr
   [junit4]   2> 122639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 122639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122643 INFO  (zkCallback-85-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1db90d08 name:ZooKeeperConnection Watcher:127.0.0.1:53827 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122650 INFO  (zkCallback-86-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1d398082 name:ZooKeeperConnection Watcher:127.0.0.1:53827/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 122653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 122656 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 122657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 122659 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 122661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 122662 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 122663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 122664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 122665 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50250_
   [junit4]   2> 122665 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50250_
   [junit4]   2> 122667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 122668 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 122669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 122669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94605713784766468-127.0.0.1:50250_-n_0000000000
   [junit4]   2> 122669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50250_
   [junit4]   2> 122670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 122671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer Overseer (id=94605713784766468-127.0.0.1:50250_-n_0000000000) starting
   [junit4]   2> 122672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 122677 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 122680 INFO  (OverseerCollectionConfigSetProcessor-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 122681 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 122681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 122713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 122713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 122715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores
   [junit4]   2> 122715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/, shard=, collection=control_collection, dataDir=data/, name=collection1, configSetProperties=configsetprops.json, coreNodeName=, transient=false}
   [junit4]   2> 122716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/
   [junit4]   2> 122716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 122717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 122717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:50250_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 122717 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 122717 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 122719 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:50250",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:50250_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 122719 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 122719 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:50250",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:50250_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 122720 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 122720 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 122722 INFO  (zkCallback-86-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 123720 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 123720 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 123720 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 123720 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 123721 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 123721 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/'
   [junit4]   2> 123736 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 123743 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 123759 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 123787 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 123788 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 123796 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 124071 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 124081 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 124083 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 124106 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 124110 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 124113 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 124114 WARN  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 124114 WARN  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 124114 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 124115 WARN  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 124115 WARN  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 124116 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 124116 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 124117 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 124117 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 124117 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@66746531
   [junit4]   2> 124117 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/data
   [junit4]   2> 124118 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/data/index/
   [junit4]   2> 124118 WARN  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 124118 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/data/index
   [junit4]   2> 124118 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=6.0966796875, floorSegmentMB=1.193359375, forceMergeDeletesPctAllowed=17.71058971923877, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3884626154644303
   [junit4]   2> 124119 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2b3c1532 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@75939bca),segFN=segments_1,generation=1}
   [junit4]   2> 124119 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 124119 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/data/
   [junit4]   2> 124120 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 124129 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 124129 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 124129 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 124129 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 124130 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 124130 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 124130 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 124130 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 124130 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 124133 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124135 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124136 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124138 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124143 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/plugins,/admin/logging,/admin/ping,/replication,/update,/admin/file,/admin/mbeans,/admin/system,/get,/update/csv,/update/json/docs,/admin/segments,/update/json,/schema,/config,standard,/admin/properties,/admin/luke
   [junit4]   2> 124145 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 124145 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 124145 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 124146 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 124146 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 124147 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=0.3056640625, floorSegmentMB=1.5283203125, forceMergeDeletesPctAllowed=18.962816152482954, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1739528631627314
   [junit4]   2> 124147 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2b3c1532 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@75939bca),segFN=segments_1,generation=1}
   [junit4]   2> 124147 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 124147 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6dfe4c3b[collection1] main
   [junit4]   2> 124147 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 124149 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 124149 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 124149 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 124149 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 124149 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 124150 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 124150 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 124150 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 124150 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 124151 INFO  (searcherExecutor-522-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6dfe4c3b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 124151 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 124151 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 124151 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 124151 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513691422834819072
   [junit4]   2> 124153 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1513691422834819072
   [junit4]   2> 124153 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 124154 INFO  (coreLoadExecutor-521-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 124155 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:50250 collection:control_collection shard:shard1
   [junit4]   2> 124156 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 124162 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94605713784766468-core_node1-n_0000000000
   [junit4]   2> 124162 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 124164 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 124164 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 124164 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50250/collection1/
   [junit4]   2> 124164 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 124164 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:50250/collection1/ has no replicas
   [junit4]   2> 124164 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50250/collection1/ shard1
   [junit4]   2> 124165 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 124165 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 124168 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 124171 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:50250",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 124218 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 124228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124230 INFO  (zkCallback-88-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5af78cc5 name:ZooKeeperConnection Watcher:127.0.0.1:53827/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 124230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 124235 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 124274 INFO  (zkCallback-88-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 124274 INFO  (zkCallback-86-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 124280 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:50250/collection1/ and leader is http://127.0.0.1:50250/collection1/
   [junit4]   2> 124280 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50250
   [junit4]   2> 124281 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 124281 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 124281 INFO  (coreZkRegister-515-thread-1-processing-x:collection1 c:control_collection r:core_node1 n:127.0.0.1:50250_ s:shard1) [n:127.0.0.1:50250_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 124284 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:50250",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:50250_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 124284 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:50250",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:50250_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 124388 INFO  (zkCallback-88-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 124388 INFO  (zkCallback-86-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 124611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1
   [junit4]   2> 124612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001
   [junit4]   2> 124613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 124620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1509144e{/,null,AVAILABLE}
   [junit4]   2> 124620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@71417ac{HTTP/1.1}{127.0.0.1:47850}
   [junit4]   2> 124620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.e.j.s.Server Started @127828ms
   [junit4]   2> 124621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=47850, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/tempDir-001/jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 124622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 124622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/'
   [junit4]   2> 124640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 124645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124647 INFO  (zkCallback-89-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1c23d8d0 name:ZooKeeperConnection Watcher:127.0.0.1:53827/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 124654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 124654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/solr.xml
   [junit4]   2> 124676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores
   [junit4]   2> 124676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1422785170
   [junit4]   2> 124676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/]
   [junit4]   2> 124676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/lib
   [junit4]   2> 124676 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/lib).
   [junit4]   2> 124684 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 124685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 124686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 124686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 124686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53827/solr
   [junit4]   2> 124686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 124686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 124687 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124688 INFO  (zkCallback-91-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5422cffc name:ZooKeeperConnection Watcher:127.0.0.1:53827 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124688 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124689 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 124694 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124695 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c22ce81 name:ZooKeeperConnection Watcher:127.0.0.1:53827/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124700 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 125706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47850_
   [junit4]   2> 125707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47850_
   [junit4]   2> 125708 INFO  (zkCallback-86-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 125708 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 125708 INFO  (zkCallback-88-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 125710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 125710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94605713784766472-127.0.0.1:47850_-n_0000000001
   [junit4]   2> 125711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94605713784766468-127.0.0.1:50250_-n_0000000000 to know if I could be the leader
   [junit4]   2> 125731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 125731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 125733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores
   [junit4]   2> 125734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, name=collection1, collection=collection1, configSetProperties=configsetprops.json, coreNodeName=, loadOnStartup=true, config=solrconfig.xml, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/, dataDir=data/}
   [junit4]   2> 125734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/
   [junit4]   2> 125734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 125736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 125736 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 125736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[55BD0976D413947D]) [n:127.0.0.1:47850_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 125736 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 125738 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 125738 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:47850",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:47850_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 125739 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:47850",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:47850_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 125739 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 125739 INFO  (OverseerStateUpdate-94605713784766468-127.0.0.1:50250_-n_0000000000) [n:127.0.0.1:50250_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 125846 INFO  (zkCallback-88-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 125846 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 125846 INFO  (zkCallback-86-thread-1-processing-n:127.0.0.1:50250_) [n:127.0.0.1:50250_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 126739 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 126739 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 126740 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 126740 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 126740 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 126741 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/'
   [junit4]   2> 126754 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 126762 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 126783 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 126836 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 126837 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 126847 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 127136 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 127148 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 127150 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 127168 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 127172 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 127175 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 127176 WARN  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 127176 WARN  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 127176 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 127176 WARN  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 127176 WARN  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 127177 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 127178 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 127178 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 127178 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 127178 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@66746531
   [junit4]   2> 127179 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/data
   [junit4]   2> 127179 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 127179 WARN  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 127179 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 127179 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=6.0966796875, floorSegmentMB=1.193359375, forceMergeDeletesPctAllowed=17.71058971923877, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3884626154644303
   [junit4]   2> 127180 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@189572bb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7b9f5f98),segFN=segments_1,generation=1}
   [junit4]   2> 127180 INFO  (coreLoadExecutor-532-thread-1-processing-n:127.0.0.1:47850_) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 127181 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47850_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_55BD0976D413947D-001/shard-1-001/cores/collection1/data/
  

[...truncated too long message...]

tric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 1374794 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[55BD0976D413947D]-worker) [n:127.0.0.1:58957_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 29, 2015 7:38:04 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {_version_=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=PostingsFormat(name=Direct), range_facet_l_dv=FSTOrd50, other_tl1=PostingsFormat(name=LuceneFixedGap), a_i1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=FSTOrd50, timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), intDefault=PostingsFormat(name=LuceneFixedGap), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Direct)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=no_NO_NY, timezone=Pacific/Efate
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=5,free=106976768,total=523763712
   [junit4]   2> NOTE: All tests run in this JVM: [DOMUtilTest, TestEmbeddedSolrServerConstructors, TestCoreContainer, FastVectorHighlighterTest, DateRangeFieldTest, SliceStateTest, TestConfigSets, TestUniqueKeyFieldResource, IgnoreCommitOptimizeUpdateProcessorFactoryTest, URLClassifyProcessorTest, MultiTermTest, PreAnalyzedFieldTest, TestFieldCollectionResource, TestCustomDocTransformer, CollectionStateFormat2Test, TestDistributedGrouping, TestCustomSort, NumericFieldsTest, TestConfigSetImmutable, CollectionsAPIDistributedZkTest, TestCursorMarkWithoutUniqueKey, TestManagedResourceStorage, TestHashQParserPlugin, TestCloudManagedSchema, AnalyticsMergeStrategyTest, TestInitParams, DistributedQueueTest, TestHashPartitioner, TestPivotHelperCode, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [153/539] on J1 in 694.71s, 1 test, 1 error <<< FAILURES!

[...truncated 1254 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 2 errors, 29 ignored (17 assumptions) [seed: 55BD0976D413947D]

Total time: 194 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1463462927 bytes
Compression is 0.0%
Took 4 min 0 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 970 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/970/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=14692, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14692, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:48734: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([76E638E196C89A23]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11311 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/init-core-data-001
   [junit4]   2> 1517093 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1517094 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1517133 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1517144 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1517148 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1517170 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_34175_hdfs____.1x4tjc/webapp
   [junit4]   2> 1517297 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1517634 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34175
   [junit4]   2> 1517714 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1517715 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1517728 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_56528_datanode____.h1snxb/webapp
   [junit4]   2> 1517870 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1518205 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56528
   [junit4]   2> 1518272 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1518274 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1518322 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58127_datanode____d8ys62/webapp
   [junit4]   2> 1518398 INFO  (IPC Server handler 4 on 54050) [    ] BlockStateChange BLOCK* processReport: from storage DS-c52156ad-5d88-454f-ad24-ff94996ea49c node DatanodeRegistration(127.0.0.1, datanodeUuid=91e59a88-932f-4ae7-825e-72186d9ab81a, infoPort=56528, ipcPort=58628, storageInfo=lv=-56;cid=testClusterID;nsid=1726480293;c=0), blocks: 0, hasStaleStorages: true, processing time: 2 msecs
   [junit4]   2> 1518398 INFO  (IPC Server handler 4 on 54050) [    ] BlockStateChange BLOCK* processReport: from storage DS-47ed38e8-280e-4d4d-9dfd-f2b11544a6cf node DatanodeRegistration(127.0.0.1, datanodeUuid=91e59a88-932f-4ae7-825e-72186d9ab81a, infoPort=56528, ipcPort=58628, storageInfo=lv=-56;cid=testClusterID;nsid=1726480293;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1518483 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1518773 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58127
   [junit4]   2> 1518994 INFO  (IPC Server handler 5 on 54050) [    ] BlockStateChange BLOCK* processReport: from storage DS-1388c3c7-2d5a-49f7-b079-b884a20b48b2 node DatanodeRegistration(127.0.0.1, datanodeUuid=05309fcc-b92e-4f06-85cf-df1283ed1c71, infoPort=58127, ipcPort=39149, storageInfo=lv=-56;cid=testClusterID;nsid=1726480293;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1518994 INFO  (IPC Server handler 5 on 54050) [    ] BlockStateChange BLOCK* processReport: from storage DS-504fd648-bf51-444c-af0b-0dbca3c93e20 node DatanodeRegistration(127.0.0.1, datanodeUuid=05309fcc-b92e-4f06-85cf-df1283ed1c71, infoPort=58127, ipcPort=39149, storageInfo=lv=-56;cid=testClusterID;nsid=1726480293;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1519120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1519120 INFO  (Thread-7086) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1519120 INFO  (Thread-7086) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1519220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.ZkTestServer start zk server on port:57722
   [junit4]   2> 1519221 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1519222 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1519228 INFO  (zkCallback-1799-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40a08af6 name:ZooKeeperConnection Watcher:127.0.0.1:57722 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1519229 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1519229 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1519229 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1519238 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1519239 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1519241 INFO  (zkCallback-1800-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55ad1e30 name:ZooKeeperConnection Watcher:127.0.0.1:57722/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1519241 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1519241 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1519241 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1519244 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1519246 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1519257 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1519259 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1519259 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1519275 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1519275 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1519280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1519280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1519290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1519290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1519297 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1519297 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1519300 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1519300 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1519306 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1519306 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1519309 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1519309 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1519312 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1519312 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1519315 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1519315 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1519318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1519318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1519661 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores/collection1
   [junit4]   2> 1519664 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1519680 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@273a34d6{/,null,AVAILABLE}
   [junit4]   2> 1519681 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2dd12771{HTTP/1.1}{127.0.0.1:54121}
   [junit4]   2> 1519681 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.Server Started @1522982ms
   [junit4]   2> 1519681 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=hdfs://localhost:54050/hdfs__localhost_54050__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores, hostPort=54121}
   [junit4]   2> 1519681 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1519681 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/'
   [junit4]   2> 1519701 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1519702 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1519704 INFO  (zkCallback-1801-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@599889a5 name:ZooKeeperConnection Watcher:127.0.0.1:57722/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1519704 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1519704 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1519706 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1519706 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/solr.xml
   [junit4]   2> 1519720 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores
   [junit4]   2> 1519720 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoreContainer New CoreContainer 1683881154
   [junit4]   2> 1519720 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/]
   [junit4]   2> 1519720 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/lib
   [junit4]   2> 1519721 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/lib).
   [junit4]   2> 1519730 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1519731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1519731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1519731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1519732 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57722/solr
   [junit4]   2> 1519732 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1519732 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1519733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1519733 INFO  (zkCallback-1803-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79173c4f name:ZooKeeperConnection Watcher:127.0.0.1:57722 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1519734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1519735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1519737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1519738 INFO  (zkCallback-1804-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@789e883e name:ZooKeeperConnection Watcher:127.0.0.1:57722/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1519738 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1519739 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1519741 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1519746 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1519748 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1519750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1519752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1519754 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1519755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1519757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1519758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54121_
   [junit4]   2> 1519759 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54121_
   [junit4]   2> 1519761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1519763 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1519765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1519766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94600192851378180-127.0.0.1:54121_-n_0000000000
   [junit4]   2> 1519767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54121_
   [junit4]   2> 1519767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1519769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer Overseer (id=94600192851378180-127.0.0.1:54121_-n_0000000000) starting
   [junit4]   2> 1519778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1519797 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1519798 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1519799 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1519800 INFO  (OverseerCollectionConfigSetProcessor-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1519807 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1519839 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1519839 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1519840 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores
   [junit4]   2> 1519842 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, shard=, config=solrconfig.xml, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores/collection1, name=collection1, collection=control_collection, transient=false, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores/collection1/, loadOnStartup=true, dataDir=data/}
   [junit4]   2> 1519842 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores/collection1/
   [junit4]   2> 1519842 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1519845 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1519845 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1519846 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1519846 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:54121_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1519853 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:54121",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:54121_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1519853 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1519854 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:54121",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:54121_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1519854 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1519855 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1519859 INFO  (zkCallback-1804-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1519860 INFO  (zkCallback-1804-thread-2-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1520854 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1520854 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1520855 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1520855 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1520855 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1520856 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores/collection1/'
   [junit4]   2> 1520875 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1520881 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1520895 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1520959 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1520961 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1520967 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1521244 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1521260 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1521262 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1521291 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1521298 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1521303 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1521306 WARN  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1521306 WARN  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1521306 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1521307 WARN  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1521307 WARN  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1521308 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1521308 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1521308 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1521309 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:54050/solr_hdfs_home
   [junit4]   2> 1521309 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1521309 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1521309 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e4cf6b
   [junit4]   2> 1521309 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1521329 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 1521329 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1521332 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1521343 WARN  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:54050/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1521343 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1521356 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 1521357 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1521357 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=84.076171875, floorSegmentMB=0.451171875, forceMergeDeletesPctAllowed=18.526214344662442, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1521398 INFO  (IPC Server handler 4 on 54050) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39536 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-c52156ad-5d88-454f-ad24-ff94996ea49c:NORMAL:127.0.0.1:45243|RBW], ReplicaUnderConstruction[[DISK]DS-504fd648-bf51-444c-af0b-0dbca3c93e20:NORMAL:127.0.0.1:39536|FINALIZED]]} size 0
   [junit4]   2> 1521403 INFO  (IPC Server handler 5 on 54050) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45243 is added to blk_1073741825_1001 size 71
   [junit4]   2> 1521407 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b870ba50 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7707791a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1521407 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1521409 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:54050/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1521413 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1521414 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1521414 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1521414 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1521414 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1521414 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1521415 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1521415 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1521415 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1521418 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1521420 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1521422 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1521423 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1521431 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/admin/segments,/schema,/admin/file,/update,/admin/properties,/admin/threads,/admin/plugins,/update/json,/update/csv,/replication,/update/json/docs,/config,/admin/ping,/get,/admin/mbeans,standard,/admin/system,/admin/luke
   [junit4]   2> 1521433 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1521433 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1521433 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 1521448 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1521448 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1521450 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@63561f07
   [junit4]   2> 1521459 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b870ba50 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7707791a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1521459 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1521459 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@33884d91[collection1] main
   [junit4]   2> 1521460 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1521461 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1521461 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1521461 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1521461 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1521461 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1521462 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1521462 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1521462 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1521462 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1521466 INFO  (searcherExecutor-3677-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33884d91[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1521478 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1521478 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 1521479 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1521479 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513603088093544448
   [junit4]   2> 1521480 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1513603088093544448
   [junit4]   2> 1521480 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1521481 INFO  (coreLoadExecutor-3676-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1521481 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:54121 collection:control_collection shard:shard1
   [junit4]   2> 1521482 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1521487 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94600192851378180-core_node1-n_0000000000
   [junit4]   2> 1521488 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1521489 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1521489 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1521489 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1521490 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54121/collection1/
   [junit4]   2> 1521490 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1521490 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:54121/collection1/ has no replicas
   [junit4]   2> 1521490 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54121/collection1/ shard1
   [junit4]   2> 1521491 INFO  (zkCallback-1804-thread-2-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1521491 INFO  (zkCallback-1804-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1521492 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1521498 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1521505 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54121",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1521610 INFO  (zkCallback-1804-thread-2-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1521610 INFO  (zkCallback-1804-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1521657 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:54121/collection1/ and leader is http://127.0.0.1:54121/collection1/
   [junit4]   2> 1521657 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54121
   [junit4]   2> 1521657 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1521657 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1521658 INFO  (coreZkRegister-3670-thread-1-processing-c:control_collection x:collection1 s:shard1 n:127.0.0.1:54121_ r:core_node1) [n:127.0.0.1:54121_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1521660 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:54121",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:54121_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1521660 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:54121",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:54121_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1521763 INFO  (zkCallback-1804-thread-2-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1521763 INFO  (zkCallback-1804-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1521848 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1521849 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1521850 INFO  (zkCallback-1806-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3f521544 name:ZooKeeperConnection Watcher:127.0.0.1:57722/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1521850 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1521851 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1521851 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1521857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1522256 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores/collection1
   [junit4]   2> 1522256 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001
   [junit4]   2> 1522257 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1522263 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@452227e5{/,null,AVAILABLE}
   [junit4]   2> 1522263 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.ServerConnector Started ServerConnector@45694ed8{HTTP/1.1}{127.0.0.1:48734}
   [junit4]   2> 1522263 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.e.j.s.Server Started @1525564ms
   [junit4]   2> 1522263 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=48734, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores, hostContext=/, solr.data.dir=hdfs://localhost:54050/hdfs__localhost_54050__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001_tempDir-002_jetty1}
   [junit4]   2> 1522264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1522264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/'
   [junit4]   2> 1522287 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1522288 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1522289 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f03646c name:ZooKeeperConnection Watcher:127.0.0.1:57722/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1522289 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1522289 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1522290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1522290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/solr.xml
   [junit4]   2> 1522304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores
   [junit4]   2> 1522304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoreContainer New CoreContainer 1570973504
   [junit4]   2> 1522304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/]
   [junit4]   2> 1522304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/lib
   [junit4]   2> 1522304 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/lib).
   [junit4]   2> 1522317 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1522318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1522318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1522319 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1522319 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57722/solr
   [junit4]   2> 1522319 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1522319 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1522323 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1522325 INFO  (zkCallback-1809-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@9f9f520 name:ZooKeeperConnection Watcher:127.0.0.1:57722 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1522326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1522326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1522329 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1522332 INFO  (zkCallback-1810-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@d3511f3 name:ZooKeeperConnection Watcher:127.0.0.1:57722/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1522332 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1522340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1523346 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48734_
   [junit4]   2> 1523347 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48734_
   [junit4]   2> 1523348 INFO  (zkCallback-1804-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1523348 INFO  (zkCallback-1810-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1523348 INFO  (zkCallback-1806-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1523348 INFO  (zkCallback-1804-thread-2-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1523350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1523351 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94600192851378184-127.0.0.1:48734_-n_0000000001
   [junit4]   2> 1523352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94600192851378180-127.0.0.1:54121_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1523398 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1523398 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1523403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores
   [junit4]   2> 1523404 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, dataDir=data/, collection=collection1, loadOnStartup=true, shard=, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores/collection1/, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores/collection1, name=collection1, config=solrconfig.xml, configSetProperties=configsetprops.json}
   [junit4]   2> 1523404 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores/collection1/
   [junit4]   2> 1523404 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1523412 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1523412 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1523414 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:48734",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:48734_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 1523415 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:48734",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:48734_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1523415 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1523415 INFO  (OverseerStateUpdate-94600192851378180-127.0.0.1:54121_-n_0000000000) [n:127.0.0.1:54121_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1523415 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1523416 INFO  (zkCallback-1806-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1523416 INFO  (zkCallback-1804-thread-1-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1523416 INFO  (zkCallback-1804-thread-2-processing-n:127.0.0.1:54121_) [n:127.0.0.1:54121_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1523417 INFO  (zkCallback-1810-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1523423 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1523423 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[76E638E196C89A23]) [n:127.0.0.1:48734_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1524417 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1524417 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1524417 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1524417 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1524418 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1524418 INFO  (coreLoadExecutor-3687-thread-1-processing-n:127.0.0.1:48734_) [n:127.0.0.1:48734_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_76E638E196C89A23-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1524447

[...truncated too long message...]

t com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3993403 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[76E638E196C89A23]-worker) [n:127.0.0.1:37813_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 28, 2015 8:34:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {multiDefault=FSTOrd50, a_t=FSTOrd50, other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=Lucene50(blocksize=128), range_facet_l=FSTOrd50, range_facet_i_dv=FSTOrd50, intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), a_i1=FSTOrd50, id=Lucene50(blocksize=128), timestamp=FSTOrd50, text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=es_MX, timezone=Etc/GMT+10
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=160677384,total=522715136
   [junit4]   2> NOTE: All tests run in this JVM: [TestSuggestSpellingConverter, DistributedTermsComponentTest, TestCloudManagedSchemaConcurrent, DocValuesTest, BigEndianAscendingWordSerializerTest, NotRequiredUniqueKeyTest, TestSearcherReuse, AnalyticsQueryTest, TestSolrCoreProperties, TestMiniSolrCloudCluster, DOMUtilTest, HttpPartitionTest, TestHashQParserPlugin, PreAnalyzedFieldTest, ConnectionManagerTest, HdfsRecoverLeaseTest, ChaosMonkeyNothingIsSafeTest, TestRTimerTree, SolrPluginUtilsTest, SegmentsInfoRequestHandlerTest, DateRangeFieldTest, ZkCLITest, TestStressVersions, TestJmxMonitoredMap, HighlighterTest, SpellPossibilityIteratorTest, TestHashPartitioner, TestImplicitCoreProperties, SpatialFilterTest, DistributedQueueTest, SolrCmdDistributorTest, TestDistributedGrouping, AssignTest, MinimalSchemaTest, RankQueryTest, TestRandomMergePolicy, TestOverriddenPrefixQueryForCustomFieldType, HdfsSyncSliceTest, TestBlobHandler, SpellCheckCollatorTest, DeleteLastCustomShardedReplicaTest, ClusterStateUpdateTest, ShardRoutingCustomTest, TermVectorComponentTest, TestRecoveryHdfs, AsyncMigrateRouteKeyTest, SimpleMLTQParserTest, NumericFieldsTest, TestSolrConfigHandler, TermsComponentTest, UniqFieldsUpdateProcessorFactoryTest, SolrInfoMBeanTest, ZkSolrClientTest, TestInitParams, TestSolrDeletionPolicy2, TestManagedSchemaFieldTypeResource, TestManagedSynonymFilterFactory, TestSurroundQueryParser, TestCustomDocTransformer, SimplePostToolTest, TestManagedResource, TestQuerySenderListener, TestNamedUpdateProcessors, DistanceFunctionTest, TestPartialUpdateDeduplication, CollectionsAPIDistributedZkTest, DistributedFacetPivotSmallTest, DistanceUnitsTest, IndexBasedSpellCheckerTest, TestHighFrequencyDictionaryFactory, TestSchemaResource, SolrCloudExampleTest, TestCodecSupport, URLClassifyProcessorTest, SoftAutoCommitTest, MBeansHandlerTest, BasicFunctionalityTest, ExitableDirectoryReaderTest, HdfsThreadLeakTest, TestBinaryResponseWriter, TestQueryUtils, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [435/539] on J1 in 2486.08s, 1 test, 1 error <<< FAILURES!

[...truncated 359 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 1 error, 29 ignored (17 assumptions) [seed: 76E638E196C89A23]

Total time: 200 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1190652711 bytes
Compression is 0.0%
Took 3 min 11 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 969 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/969/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node3:{"node_name":"127.0.0.1:48700_qzf%2Fx","core":"c8n_1x3_lf_shard1_replica3","state":"active","base_url":"http://127.0.0.1:48700/qzf/x","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf)={   "replicationFactor":"3",   "autoAddReplicas":"false",   "router":{"name":"compositeId"},   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node1":{           "node_name":"127.0.0.1:46030_qzf%2Fx",           "core":"c8n_1x3_lf_shard1_replica2",           "state":"down",           "base_url":"http://127.0.0.1:46030/qzf/x"},         "core_node2":{           "state":"down",           "base_url":"http://127.0.0.1:39348/qzf/x",           "core":"c8n_1x3_lf_shard1_replica1",           "node_name":"127.0.0.1:39348_qzf%2Fx"},         "core_node3":{           "node_name":"127.0.0.1:48700_qzf%2Fx",           "core":"c8n_1x3_lf_shard1_replica3",           "state":"active",           "base_url":"http://127.0.0.1:48700/qzf/x",           "leader":"true"}}}},   "maxShardsPerNode":"1"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node3:{"node_name":"127.0.0.1:48700_qzf%2Fx","core":"c8n_1x3_lf_shard1_replica3","state":"active","base_url":"http://127.0.0.1:48700/qzf/x","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf)={
  "replicationFactor":"3",
  "autoAddReplicas":"false",
  "router":{"name":"compositeId"},
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node1":{
          "node_name":"127.0.0.1:46030_qzf%2Fx",
          "core":"c8n_1x3_lf_shard1_replica2",
          "state":"down",
          "base_url":"http://127.0.0.1:46030/qzf/x"},
        "core_node2":{
          "state":"down",
          "base_url":"http://127.0.0.1:39348/qzf/x",
          "core":"c8n_1x3_lf_shard1_replica1",
          "node_name":"127.0.0.1:39348_qzf%2Fx"},
        "core_node3":{
          "node_name":"127.0.0.1:48700_qzf%2Fx",
          "core":"c8n_1x3_lf_shard1_replica3",
          "state":"active",
          "base_url":"http://127.0.0.1:48700/qzf/x",
          "leader":"true"}}}},
  "maxShardsPerNode":"1"}
	at __randomizedtesting.SeedInfo.seed([5B9382868E538587:D3C7BD5C20AFE87F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:166)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test

Error Message:
The Monkey ran for over 30 seconds and no jetties were stopped - this is worth investigating!

Stack Trace:
java.lang.AssertionError: The Monkey ran for over 30 seconds and no jetties were stopped - this is worth investigating!
	at __randomizedtesting.SeedInfo.seed([5B9382868E538587:D3C7BD5C20AFE87F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:539)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:143)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:     1) Thread[id=12609, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)    2) Thread[id=12610, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 
   1) Thread[id=12609, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   2) Thread[id=12610, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
	at __randomizedtesting.SeedInfo.seed([5B9382868E538587]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=12610, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at java.security.AccessController.doPrivileged(Native Method)         at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:104)         at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:83)         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:457)         at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)         at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)         at java.io.PrintStream.flush(PrintStream.java:338)         at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)         at java.io.PrintStream.write(PrintStream.java:482)         at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)         at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)         at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)         at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)         at java.io.PrintStream.write(PrintStream.java:527)         at java.io.PrintStream.print(PrintStream.java:669)         at java.io.PrintStream.println(PrintStream.java:823)         at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)         at java.lang.Throwable.printStackTrace(Throwable.java:658)         at java.lang.Throwable.printStackTrace(Throwable.java:643)         at java.lang.Throwable.printStackTrace(Throwable.java:634)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:122)    2) Thread[id=12609, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:163)         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:131)         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:381)         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:297)         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=12610, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at java.security.AccessController.doPrivileged(Native Method)
        at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:104)
        at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:83)
        at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:457)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at java.io.PrintStream.flush(PrintStream.java:338)
        at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
        at java.io.PrintStream.write(PrintStream.java:482)
        at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
        at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
        at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
        at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
        at java.io.PrintStream.write(PrintStream.java:527)
        at java.io.PrintStream.print(PrintStream.java:669)
        at java.io.PrintStream.println(PrintStream.java:823)
        at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)
        at java.lang.Throwable.printStackTrace(Throwable.java:658)
        at java.lang.Throwable.printStackTrace(Throwable.java:643)
        at java.lang.Throwable.printStackTrace(Throwable.java:634)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:122)
   2) Thread[id=12609, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:163)
        at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:131)
        at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:381)
        at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:297)
        at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
        at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
	at __randomizedtesting.SeedInfo.seed([5B9382868E538587]:0)




Build Log:
[...truncated 10268 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/init-core-data-001
   [junit4]   2> 715687 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[5B9382868E538587]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /qzf/x
   [junit4]   2> 715700 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 715711 INFO  (Thread-3175) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 715711 INFO  (Thread-3175) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 715811 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.ZkTestServer start zk server on port:33383
   [junit4]   2> 715811 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 715837 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 715851 INFO  (zkCallback-1583-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d880e8b name:ZooKeeperConnection Watcher:127.0.0.1:33383 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 715851 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 715851 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 715851 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 715855 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 715871 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 715887 INFO  (zkCallback-1584-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a491cba name:ZooKeeperConnection Watcher:127.0.0.1:33383/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 715888 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 715888 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 715888 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 715890 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 715891 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 715892 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 715893 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 715893 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 715895 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 715896 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 715897 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 715897 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 715899 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 715899 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 715900 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 715900 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 715902 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 715902 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 715903 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 715903 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 715905 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 715905 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 715906 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 715907 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 715908 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 715908 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 715909 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 715909 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 716240 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1
   [junit4]   2> 716243 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 716245 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62ae2a85{/qzf/x,null,AVAILABLE}
   [junit4]   2> 716246 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1869b460{HTTP/1.1}{127.0.0.1:38766}
   [junit4]   2> 716246 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.Server Started @719106ms
   [junit4]   2> 716246 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/qzf/x, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/tempDir-001/control/data, hostPort=46030, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores}
   [junit4]   2> 716246 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7f423820
   [junit4]   2> 716246 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/'
   [junit4]   2> 716262 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 716264 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 716264 INFO  (zkCallback-1585-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1803ca9a name:ZooKeeperConnection Watcher:127.0.0.1:33383/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 716264 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 716265 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 716266 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 716266 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/solr.xml
   [junit4]   2> 716280 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores
   [junit4]   2> 716280 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoreContainer New CoreContainer 1070571671
   [junit4]   2> 716280 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/]
   [junit4]   2> 716280 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/lib
   [junit4]   2> 716280 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/lib).
   [junit4]   2> 716289 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 716290 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 716291 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 716291 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 716291 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33383/solr
   [junit4]   2> 716291 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 716291 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 716293 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 716293 INFO  (zkCallback-1587-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@64238c71 name:ZooKeeperConnection Watcher:127.0.0.1:33383 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 716294 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 716294 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 716308 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 716312 INFO  (zkCallback-1588-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57a8ed29 name:ZooKeeperConnection Watcher:127.0.0.1:33383/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 716312 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 716313 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 716315 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 716317 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 716319 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 716320 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 716322 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 716323 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 716324 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 716325 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 716326 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46030_qzf%2Fx
   [junit4]   2> 716326 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46030_qzf%2Fx
   [junit4]   2> 716328 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 716329 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 716330 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 716330 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000
   [junit4]   2> 716331 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46030_qzf%2Fx
   [junit4]   2> 716331 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 716332 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.Overseer Overseer (id=94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) starting
   [junit4]   2> 716333 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 716338 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 716451 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 716452 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 716459 INFO  (OverseerCollectionConfigSetProcessor-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 716475 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 716493 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 716493 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 716494 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores
   [junit4]   2> 716495 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1, schema=schema.xml, coreNodeName=, shard=, collection=control_collection, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/, loadOnStartup=true, config=solrconfig.xml, dataDir=data/, name=collection1, transient=false}
   [junit4]   2> 716495 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/
   [junit4]   2> 716495 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 716497 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 716497 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 716502 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 716502 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 716502 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:46030_qzf%2Fx",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46030/qzf/x"} current state version: 0
   [junit4]   2> 716502 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:46030_qzf%2Fx",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46030/qzf/x"}
   [junit4]   2> 716503 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 716503 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 716503 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 716505 INFO  (zkCallback-1588-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 716507 INFO  (zkCallback-1588-thread-2-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 717504 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 717504 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 717504 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 717504 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 717505 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 717505 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/'
   [junit4]   2> 717534 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 717541 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 717571 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 717626 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 717639 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 717646 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 717970 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 717981 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 717984 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 717997 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 718001 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 718005 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 718006 WARN  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 718007 WARN  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 718007 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 718008 WARN  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 718008 WARN  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 718010 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 718010 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 718010 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 718010 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 718011 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@59334754
   [junit4]   2> 718011 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/data
   [junit4]   2> 718011 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/data/index/
   [junit4]   2> 718012 WARN  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 718012 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/data/index
   [junit4]   2> 718012 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=8, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=76.69140625, floorSegmentMB=1.1806640625, forceMergeDeletesPctAllowed=26.41525090822941, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5618391405130249
   [junit4]   2> 718013 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@295c4f56 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6b5cac50),segFN=segments_1,generation=1}
   [junit4]   2> 718013 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 718035 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/data/
   [junit4]   2> 718035 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 718036 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 718036 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 718036 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 718036 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 718036 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 718036 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 718037 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 718037 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 718037 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 718040 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 718041 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 718042 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 718043 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 718047 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/properties,/admin/mbeans,/admin/system,/admin/threads,/get,/update/json/docs,/schema,standard,/admin/logging,/replication,/update/csv,/update,/admin/file,/update/json,/admin/plugins,/admin/luke,/admin/ping,/config,/admin/segments
   [junit4]   2> 718048 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 718049 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 718049 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 718050 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 718050 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 718050 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=42.91015625, floorSegmentMB=1.9130859375, forceMergeDeletesPctAllowed=2.2984479378997156, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3660539635179164
   [junit4]   2> 718051 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@295c4f56 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6b5cac50),segFN=segments_1,generation=1}
   [junit4]   2> 718051 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 718051 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@19614d47[collection1] main
   [junit4]   2> 718051 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 718052 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 718052 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 718053 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 718053 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 718053 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 718054 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513510669770555392
   [junit4]   2> 718057 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1513510669770555392
   [junit4]   2> 718057 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 718058 INFO  (coreLoadExecutor-2170-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 718068 INFO  (searcherExecutor-2171-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@19614d47[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 718068 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46030/qzf/x collection:control_collection shard:shard1
   [junit4]   2> 718069 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 718073 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94594416708157444-core_node1-n_0000000000
   [junit4]   2> 718073 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 718075 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 718075 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 718075 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46030/qzf/x/collection1/
   [junit4]   2> 718075 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 718075 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46030/qzf/x/collection1/ has no replicas
   [junit4]   2> 718075 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46030/qzf/x/collection1/ shard1
   [junit4]   2> 718075 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 718076 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 718077 INFO  (zkCallback-1588-thread-3-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 718077 INFO  (zkCallback-1588-thread-2-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 718082 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 718084 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46030/qzf/x",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 718187 INFO  (zkCallback-1588-thread-2-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 718188 INFO  (zkCallback-1588-thread-3-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 718236 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46030/qzf/x/collection1/ and leader is http://127.0.0.1:46030/qzf/x/collection1/
   [junit4]   2> 718236 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46030/qzf/x
   [junit4]   2> 718236 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 718236 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 718236 INFO  (coreZkRegister-2164-thread-1-processing-n:127.0.0.1:46030_qzf%2Fx s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:46030_qzf%2Fx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 718238 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:46030_qzf%2Fx",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46030/qzf/x"} current state version: 3
   [junit4]   2> 718239 INFO  (OverseerStateUpdate-94594416708157444-127.0.0.1:46030_qzf%2Fx-n_0000000000) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:46030_qzf%2Fx",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46030/qzf/x"}
   [junit4]   2> 718342 INFO  (zkCallback-1588-thread-3-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 718342 INFO  (zkCallback-1588-thread-2-processing-n:127.0.0.1:46030_qzf%2Fx) [n:127.0.0.1:46030_qzf%2Fx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 718505 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 718506 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 718508 INFO  (zkCallback-1590-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6ea06137 name:ZooKeeperConnection Watcher:127.0.0.1:33383/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718508 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 718508 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 718508 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 718510 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 718916 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/cores/collection1
   [junit4]   2> 718917 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001
   [junit4]   2> 718918 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 718963 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37d35f62{/qzf/x,null,AVAILABLE}
   [junit4]   2> 718963 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.ServerConnector Started ServerConnector@161b3790{HTTP/1.1}{127.0.0.1:49874}
   [junit4]   2> 718964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.e.j.s.Server Started @721824ms
   [junit4]   2> 718964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/tempDir-001/jetty1, hostContext=/qzf/x, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/cores, solrconfig=solrconfig.xml, hostPort=48700}
   [junit4]   2> 718964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7f423820
   [junit4]   2> 718964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/'
   [junit4]   2> 718980 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 719002 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 719019 INFO  (zkCallback-1591-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@cc3a606 name:ZooKeeperConnection Watcher:127.0.0.1:33383/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 719019 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 719019 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 719021 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 719021 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/solr.xml
   [junit4]   2> 719047 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/cores
   [junit4]   2> 719048 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoreContainer New CoreContainer 359189798
   [junit4]   2> 719048 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/]
   [junit4]   2> 719048 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/lib
   [junit4]   2> 719048 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_5B9382868E538587-001/shard-1-001/lib).
   [junit4]   2> 719069 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 719070 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 719071 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 719071 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 719071 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33383/solr
   [junit4]   2> 719071 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 719071 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 719095 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 719122 INFO  (zkCallback-1593-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57656bd3 name:ZooKeeperConnection Watcher:127.0.0.1:33383 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 719123 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 719123 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 719140 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:48700_qzf%2Fx    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 719159 INFO  (zkCallback-1594-thread-1-processing-n:127.0.0.1:48700_qzf%2Fx) [n:127.0.0.1:48700_qzf%2Fx    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36d03059 name:ZooKeeperConnection Watcher:127.0.0.1:33383/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 719159 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:48700_qzf%2Fx    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 719164 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[5B9382868E538587]) [n:127.0.0.1:48700_qzf%2Fx    ] o.a.s.c.c.ZkStateReader Upda

[...truncated too long message...]

ne-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.17.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150927_191212_321.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150927_191212_321.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1505)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 133 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 872143883 bytes
Compression is 0.0%
Took 2 min 52 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 968 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/968/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=72200, name=Thread-64458, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=72200, name=Thread-64458, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
	at __randomizedtesting.SeedInfo.seed([40AF630393621DF1:C8FB5CD93D9E7009]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:51686/collection1
	at __randomizedtesting.SeedInfo.seed([40AF630393621DF1]:0)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:645)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:51686/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 5 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:42084/w/hg: Could not load collection from ZK:awholynewcollection_7

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42084/w/hg: Could not load collection from ZK:awholynewcollection_7
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=13335, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13335, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:53175: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([40AF630393621DF1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10772 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/init-core-data-001
   [junit4]   2> 2083947 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 2083947 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2084072 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2084093 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2084098 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2084120 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_60932_hdfs____.i0wph4/webapp
   [junit4]   2> 2084595 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2085043 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60932
   [junit4]   2> 2085466 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2085468 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2085487 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37044_datanode____kv3837/webapp
   [junit4]   2> 2085697 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2086553 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37044
   [junit4]   2> 2086835 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2086839 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2086890 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_48180_datanode____.pby304/webapp
   [junit4]   2> 2087135 INFO  (IPC Server handler 3 on 46071) [    ] BlockStateChange BLOCK* processReport: from storage DS-70b5ddb4-26dd-44be-9458-49b18b3b8018 node DatanodeRegistration(127.0.0.1, datanodeUuid=6456d89f-51b0-4444-bd10-d4d91608ab0c, infoPort=37044, ipcPort=48272, storageInfo=lv=-56;cid=testClusterID;nsid=986782969;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 2087135 INFO  (IPC Server handler 3 on 46071) [    ] BlockStateChange BLOCK* processReport: from storage DS-7dc7176a-b04c-42a7-aa92-291df459a771 node DatanodeRegistration(127.0.0.1, datanodeUuid=6456d89f-51b0-4444-bd10-d4d91608ab0c, infoPort=37044, ipcPort=48272, storageInfo=lv=-56;cid=testClusterID;nsid=986782969;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2087208 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2087769 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[40AF630393621DF1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48180
   [junit4]   2> 2088148 INFO  (IPC Server handler 9 on 46071) [    ] BlockStateChange BLOCK* processReport: from storage DS-7bb33a3c-7ef7-49e3-a9c0-a0ebfa292473 node DatanodeRegistration(127.0.0.1, datanodeUuid=e730f3a8-9f49-411c-a4a7-eca1b4589b0b, infoPort=48180, ipcPort=33023, storageInfo=lv=-56;cid=testClusterID;nsid=986782969;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2088149 INFO  (IPC Server handler 9 on 46071) [    ] BlockStateChange BLOCK* processReport: from storage DS-a8587830-cb19-496e-bb3b-ea2620cfaf4d node DatanodeRegistration(127.0.0.1, datanodeUuid=e730f3a8-9f49-411c-a4a7-eca1b4589b0b, infoPort=48180, ipcPort=33023, storageInfo=lv=-56;cid=testClusterID;nsid=986782969;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2088280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2088281 INFO  (Thread-4694) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2088281 INFO  (Thread-4694) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2088381 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.ZkTestServer start zk server on port:51993
   [junit4]   2> 2088381 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2088395 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2088413 INFO  (zkCallback-1562-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74f829c9 name:ZooKeeperConnection Watcher:127.0.0.1:51993 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2088415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2088416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2088416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2088420 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2088435 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2088449 INFO  (zkCallback-1563-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d4283c6 name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2088449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2088449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2088449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2088451 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2088452 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2088453 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2088455 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2088455 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2088457 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2088457 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2088459 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2088459 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2088460 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2088460 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2088462 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2088462 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2088463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2088463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2088465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2088465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2088466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2088466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2088467 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2088468 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2088469 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2088469 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2088470 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2088470 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2088473 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2088487 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2088496 INFO  (zkCallback-1564-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e2dfa4a name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2088497 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2088497 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2089381 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores/collection1
   [junit4]   2> 2089391 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2089428 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@b6adc9a{/,null,AVAILABLE}
   [junit4]   2> 2089434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7fd10877{SSL-http/1.1}{127.0.0.1:36736}
   [junit4]   2> 2089434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.Server Started @2092465ms
   [junit4]   2> 2089434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores, hostPort=36736, solr.data.dir=hdfs://localhost:46071/hdfs__localhost_46071__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001_tempDir-002_control_data, hostContext=/}
   [junit4]   2> 2089434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2089434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/'
   [junit4]   2> 2089550 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2089554 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2089588 INFO  (zkCallback-1565-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62bb21bd name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2089588 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2089588 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2089592 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2089592 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/solr.xml
   [junit4]   2> 2089632 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores
   [junit4]   2> 2089632 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1759432952
   [junit4]   2> 2089632 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/]
   [junit4]   2> 2089632 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/lib
   [junit4]   2> 2089633 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/lib).
   [junit4]   2> 2089657 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2089666 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2089669 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2089669 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2089669 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51993/solr
   [junit4]   2> 2089669 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2089669 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2089675 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2089682 INFO  (zkCallback-1567-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57c0e97d name:ZooKeeperConnection Watcher:127.0.0.1:51993 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2089683 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2089683 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2089688 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2089691 INFO  (zkCallback-1568-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27eac933 name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2089691 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2089697 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2089702 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2089710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2089715 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2089718 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2089721 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2089723 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2089724 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2089726 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2089727 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36736_
   [junit4]   2> 2089728 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36736_
   [junit4]   2> 2089730 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2089731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2089733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2089734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94589629927260165-127.0.0.1:36736_-n_0000000000
   [junit4]   2> 2089734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36736_
   [junit4]   2> 2089734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2089735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer Overseer (id=94589629927260165-127.0.0.1:36736_-n_0000000000) starting
   [junit4]   2> 2089738 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2089749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2089750 INFO  (OverseerCollectionConfigSetProcessor-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2089750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2089752 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2089812 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2089812 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2089813 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores
   [junit4]   2> 2089814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores/collection1, coreNodeName=, schema=schema.xml, transient=false, loadOnStartup=true, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores/collection1/, dataDir=data/, shard=, collection=control_collection, name=collection1}
   [junit4]   2> 2089814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores/collection1/
   [junit4]   2> 2089814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2089816 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2089816 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2089818 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2089820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2089820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:36736_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2089821 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36736_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"https://127.0.0.1:36736",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 2089821 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:36736_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"https://127.0.0.1:36736",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 2089822 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2089822 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2089824 INFO  (zkCallback-1568-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2090818 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2090819 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2090819 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2090819 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2090820 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2090820 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores/collection1/'
   [junit4]   2> 2090844 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2090856 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2090889 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2091023 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2091040 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2091061 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2092126 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2092142 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2092147 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2092159 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2092162 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2092171 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2092172 WARN  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2092172 WARN  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2092172 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2092173 WARN  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2092173 WARN  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2092175 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2092175 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2092175 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 2092197 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:46071/solr_hdfs_home
   [junit4]   2> 2092197 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2092197 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2092197 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@147f04fb
   [junit4]   2> 2092197 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2092327 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 2092327 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2092335 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 2092436 WARN  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:46071/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2092436 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2092486 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 2092486 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2092487 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2092606 INFO  (IPC Server handler 2 on 46071) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44162 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-70b5ddb4-26dd-44be-9458-49b18b3b8018:NORMAL:127.0.0.1:57553|RBW], ReplicaUnderConstruction[[DISK]DS-a8587830-cb19-496e-bb3b-ea2620cfaf4d:NORMAL:127.0.0.1:44162|FINALIZED]]} size 0
   [junit4]   2> 2092609 INFO  (IPC Server handler 3 on 46071) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57553 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a8587830-cb19-496e-bb3b-ea2620cfaf4d:NORMAL:127.0.0.1:44162|FINALIZED], ReplicaUnderConstruction[[DISK]DS-7dc7176a-b04c-42a7-aa92-291df459a771:NORMAL:127.0.0.1:57553|FINALIZED]]} size 0
   [junit4]   2> 2092621 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7836ff1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44f7416a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2092621 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2092624 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:46071/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 2092628 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2092628 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2092629 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2092629 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2092629 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2092629 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2092629 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2092629 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2092629 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2092636 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2092638 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2092639 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2092640 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2092651 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/update/csv,/admin/segments,/get,/update/json,/admin/luke,standard,/admin/threads,/admin/mbeans,/schema,/config,/update/json/docs,/admin/file,/replication,/update,/admin/properties,/admin/ping,/admin/system,/admin/logging
   [junit4]   2> 2092653 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2092677 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2092677 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 2092697 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2092697 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2092707 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2092901 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7836ff1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44f7416a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2092901 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2092902 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@630e3ea3[collection1] main
   [junit4]   2> 2092902 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2092903 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2092903 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2092904 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2092904 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2092904 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2092905 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2092905 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2092905 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2092905 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2092906 INFO  (searcherExecutor-3866-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@630e3ea3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2092937 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2092938 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 2092938 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2092939 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513434083717283840
   [junit4]   2> 2092939 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1513434083717283840
   [junit4]   2> 2092940 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2092941 INFO  (coreLoadExecutor-3865-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2092950 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:36736 collection:control_collection shard:shard1
   [junit4]   2> 2092951 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2092970 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94589629927260165-core_node1-n_0000000000
   [junit4]   2> 2092970 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2092972 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2092972 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2092972 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:36736/collection1/
   [junit4]   2> 2092972 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2092972 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:36736/collection1/ has no replicas
   [junit4]   2> 2092972 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:36736/collection1/ shard1
   [junit4]   2> 2092972 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2092973 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2092974 INFO  (zkCallback-1568-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2092977 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2092981 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:36736",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 2093084 INFO  (zkCallback-1568-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2093131 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:36736/collection1/ and leader is https://127.0.0.1:36736/collection1/
   [junit4]   2> 2093131 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:36736
   [junit4]   2> 2093131 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2093131 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2093131 INFO  (coreZkRegister-3859-thread-1-processing-n:127.0.0.1:36736_ s:shard1 r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:36736_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2093136 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36736_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"https://127.0.0.1:36736",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 2093138 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:36736_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"https://127.0.0.1:36736",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 2093241 INFO  (zkCallback-1568-thread-1-processing-n:127.0.0.1:36736_) [n:127.0.0.1:36736_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2093334 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2093343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2093372 INFO  (zkCallback-1570-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@266c576e name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2093375 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2093376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2093376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2093378 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2093378 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2093378 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2093395 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2093395 INFO  (zkCallback-1571-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6ead0072 name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2093395 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2093396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2093398 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 2093398 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2093530 INFO  (OverseerStateUpdate-94589629927260165-127.0.0.1:36736_-n_0000000000) [n:127.0.0.1:36736_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2094389 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/cores/collection1
   [junit4]   2> 2094389 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001
   [junit4]   2> 2094390 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2094436 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7a99906c{/,null,AVAILABLE}
   [junit4]   2> 2094441 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3fb1d0eb{SSL-http/1.1}{127.0.0.1:53175}
   [junit4]   2> 2094441 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.e.j.s.Server Started @2097472ms
   [junit4]   2> 2094441 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:46071/hdfs__localhost_46071__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=53175, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/cores}
   [junit4]   2> 2094442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2094442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/'
   [junit4]   2> 2094731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2094732 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2094734 INFO  (zkCallback-1572-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2e87252e name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2094734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2094735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2094737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2094737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/solr.xml
   [junit4]   2> 2094737 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x1500cc46b400008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2094753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/cores
   [junit4]   2> 2094753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoreContainer New CoreContainer 421590970
   [junit4]   2> 2094753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/]
   [junit4]   2> 2094753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/lib
   [junit4]   2> 2094753 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_40AF630393621DF1-001/shard-1-001/lib).
   [junit4]   2> 2094762 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2094764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2094766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2094766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2094766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51993/solr
   [junit4]   2> 2094766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2094766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2094768 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2094769 INFO  (zkCallback-1574-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@407eeb69 name:ZooKeeperConnection Watcher:127.0.0.1:51993 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2094770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2094771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2094773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:53175_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2094775 INFO  (zkCallback-1575-thread-1-processing-n:127.0.0.1:53175_) [n:127.0.0.1:53175_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79f576ee name:ZooKeeperConnection Watcher:127.0.0.1:51993/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2094775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:53175_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2094791 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:53175_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2095802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:53175_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53175_
   [junit4]   2> 2095803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[40AF630393621DF1]) [n:127.0.0.1:53175_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53175_
   [junit4]   2> 2095804 INFO  (zkCallback-1570-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected

[...truncated too long message...]

n5_6 s:shard5 r:core_node12 x:awholynewstresscollection_collection5_6_shard5_replica2] o.a.s.c.ZkController :org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect
   [junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:604)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:305)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:158)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:311)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:308)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:90)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:84)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:602)
   [junit4]   2> 	... 6 more
   [junit4]   2> 
   [junit4]   2> 4653683 ERROR (zkCallback-812-thread-19-processing-n:127.0.0.1:35458_w%2Fhg-EventThread) [n:127.0.0.1:35458_w%2Fhg c:awholynewstresscollection_collection5_6 s:shard5 r:core_node12 x:awholynewstresscollection_collection5_6_shard5_replica2] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:366)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:158)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect
   [junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:604)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController$1.command(ZkController.java:305)
   [junit4]   2> 	... 5 more
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:311)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:308)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:90)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:84)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:602)
   [junit4]   2> 	... 6 more
   [junit4]   2> 
   [junit4]   2> 4653683 INFO  (zkCallback-812-thread-19-processing-n:127.0.0.1:35458_w%2Fhg-EventThread) [n:127.0.0.1:35458_w%2Fhg c:awholynewstresscollection_collection5_6 s:shard5 r:core_node12 x:awholynewstresscollection_collection5_6_shard5_replica2] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 4653683 INFO  (zkCallback-812-thread-19-processing-n:127.0.0.1:35458_w%2Fhg-EventThread) [n:127.0.0.1:35458_w%2Fhg c:awholynewstresscollection_collection5_6 s:shard5 r:core_node12 x:awholynewstresscollection_collection5_6_shard5_replica2] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_i_dv=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=Asserting), text=Lucene50(blocksize=128), a_i1=PostingsFormat(name=Asserting), timestamp=PostingsFormat(name=Asserting), multiDefault=PostingsFormat(name=Asserting), id=PostingsFormat(name=Memory doPackFST= false), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= false), range_facet_l=PostingsFormat(name=Asserting)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=ms_MY, timezone=America/Caracas
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=160938600,total=532152320
   [junit4]   2> NOTE: All tests run in this JVM: [BitVectorTest, TestShortCircuitedRequests, TestElisionMultitermQuery, TestWriterPerf, DistributedSuggestComponentTest, TestCSVResponseWriter, TestJmxIntegration, SimpleFacetsTest, TestFuzzyAnalyzedSuggestions, TestDefaultSearchFieldResource, TestSolrQueryParserDefaultOperatorResource, AsyncCallRequestStatusResponseTest, TestNonDefinedSimilarityFactory, CursorMarkTest, TestJsonFacets, BasicAuthIntegrationTest, HdfsWriteToMultipleCollectionsTest, TestStressReorder, TestSolrQueryParser, CoreAdminHandlerTest, TestCopyFieldCollectionResource, TestExactSharedStatsCache, TestBulkSchemaAPI, DirectUpdateHandlerOptimizeTest, BadIndexSchemaTest, UpdateParamsTest, TestFastLRUCache, CurrencyFieldXmlFileTest, TestMiniSolrCloudClusterBase, TestLeaderElectionZkExpiry, DistribDocExpirationUpdateProcessorTest, TermVectorComponentDistributedTest, BinaryUpdateRequestHandlerTest, TestPostingsSolrHighlighter, SuggesterTest, SparseHLLTest, CoreAdminCreateDiscoverTest, TestClassNameShortening, TestRemoveLastDynamicCopyField, TestReplicaProperties, TestReplicationHandlerBackup, SuggesterTSTTest, BJQParserTest, TestCollapseQParserPlugin, TestGroupingSearch, TestSolrDynamicMBean, DebugComponentTest, TestSearchPerf, TestPHPSerializedResponseWriter, RulesTest, DistributedQueryComponentCustomSortTest, TestSort, TestCollationField, SearchHandlerTest, TestJettySolrRunner, RequestLoggingTest, RecoveryZkTest, TestSolrCloudWithKerberosAlt, TestConfigSetsAPI, TestCharFilters, TestSolrIndexConfig, OverseerCollectionConfigSetProcessorTest, NoCacheHeaderTest, TestNRTOpen, TestSmileRequest, BadComponentTest, PluginInfoTest, FieldMutatingUpdateProcessorTest, TestReRankQParserPlugin, TestDefaultStatsCache, TestFieldResource, QueryParsingTest, DirectSolrConnectionTest, TestAnalyzeInfixSuggestions, DistributedFacetPivotLongTailTest, TestNoOpRegenerator, LeaderElectionTest, TestLeaderInitiatedRecoveryThread, TestReversedWildcardFilterFactory, TestSweetSpotSimilarityFactory, AutoCommitTest, TestManagedSchemaFieldResource, SampleTest, TestStandardQParsers, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [436/539] on J2 in 3716.88s, 1 test, 1 error <<< FAILURES!

[...truncated 361 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 3 errors, 29 ignored (17 assumptions) [seed: 40AF630393621DF1]

Total time: 214 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1294378618 bytes
Compression is 0.0%
Took 3 min 30 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 967 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/967/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.DistributedVersionInfoTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([49DB32BD14223847]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DistributedVersionInfoTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([49DB32BD14223847]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=11491, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11491, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41333/j/ki: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([49DB32BD14223847]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=121369, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=121369, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56602: Could not find collection : awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([49DB32BD14223847]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10713 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/init-core-data-001
   [junit4]   2> 2477474 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[49DB32BD14223847]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2477474 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[49DB32BD14223847]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /j/ki
   [junit4]   2> 2477478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2477479 INFO  (Thread-4476) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2477479 INFO  (Thread-4476) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2477579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.ZkTestServer start zk server on port:45835
   [junit4]   2> 2477579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2477580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2477582 INFO  (zkCallback-1292-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a447689 name:ZooKeeperConnection Watcher:127.0.0.1:45835 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2477582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2477583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2477583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2477585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2477586 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2477587 INFO  (zkCallback-1293-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@735c5e30 name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2477587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2477587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2477587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2477589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2477591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2477592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2477594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2477594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2477597 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2477597 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2477600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2477600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2477602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2477602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2477604 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2477604 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2477606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2477606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2477608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2477608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2477610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2477610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2477612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2477612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2477614 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2477615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2477617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2477617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2477619 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2477620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2477621 INFO  (zkCallback-1294-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c5b356c name:ZooKeeperConnection Watcher:127.0.0.1:45835 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2477621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2477622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2477622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2477624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2477625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2477626 INFO  (zkCallback-1295-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4bf9f29f name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2477626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2477626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2477627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 2477627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 2477629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 2477629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 2477632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 2477632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 2477634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2477634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2477636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 2477636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 2477638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 2477638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 2477640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 2477640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 2477642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2477642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2477644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2477644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2477646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 2477646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 2477648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 2477648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 2477650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 2477650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 2477922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1
   [junit4]   2> 2477924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2477926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c4cf7e6{/j/ki,null,AVAILABLE}
   [junit4]   2> 2477934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68ecce53{HTTP/1.1}{127.0.0.1:38471}
   [junit4]   2> 2477934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.Server Started @2480935ms
   [junit4]   2> 2477934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/tempDir-001/control/data, hostPort=38471, hostContext=/j/ki, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores}
   [junit4]   2> 2477935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2477935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/'
   [junit4]   2> 2477951 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2477951 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2477952 INFO  (zkCallback-1296-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56e28537 name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2477953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2477953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2477954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2477954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/solr.xml
   [junit4]   2> 2477968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores
   [junit4]   2> 2477968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoreContainer New CoreContainer 1684831121
   [junit4]   2> 2477968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/]
   [junit4]   2> 2477968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/lib
   [junit4]   2> 2477968 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/lib).
   [junit4]   2> 2477977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2477978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2477979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2477979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2477979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45835/solr
   [junit4]   2> 2477979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2477979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2477980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2477981 INFO  (zkCallback-1298-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10b72283 name:ZooKeeperConnection Watcher:127.0.0.1:45835 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2477982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2477982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2477986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2477987 INFO  (zkCallback-1299-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@678b6299 name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2477987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2477988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2477990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2477993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2477995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2477997 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2477999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2478001 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2478002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2478003 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2478004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38471_j%2Fki
   [junit4]   2> 2478004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38471_j%2Fki
   [junit4]   2> 2478006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2478007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2478008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2478009 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000
   [junit4]   2> 2478009 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38471_j%2Fki
   [junit4]   2> 2478009 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2478010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer Overseer (id=94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) starting
   [junit4]   2> 2478012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2478019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2478019 INFO  (OverseerCollectionConfigSetProcessor-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2478020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2478020 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2478042 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2478043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2478043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores
   [junit4]   2> 2478044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, configSetProperties=configsetprops.json, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/, loadOnStartup=true, schema=schema.xml, transient=false, name=collection1, coreNodeName=, config=solrconfig.xml}
   [junit4]   2> 2478044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/
   [junit4]   2> 2478044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2478045 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2478045 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2478045 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2478045 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2478047 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:38471_j%2Fki",
   [junit4]   2>   "base_url":"http://127.0.0.1:38471/j/ki"} current state version: 0
   [junit4]   2> 2478047 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2478047 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:38471_j%2Fki",
   [junit4]   2>   "base_url":"http://127.0.0.1:38471/j/ki"}
   [junit4]   2> 2478048 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2478048 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2478050 INFO  (zkCallback-1299-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2479048 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2479048 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2479048 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2479049 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2479049 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2479049 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/'
   [junit4]   2> 2479064 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2479069 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2479094 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2479122 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2479123 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2479130 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2479364 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2479371 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2479373 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2479382 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2479385 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2479387 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2479388 WARN  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2479388 WARN  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2479388 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2479389 WARN  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2479389 WARN  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2479390 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2479390 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2479390 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2479390 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2479391 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@56ab5730
   [junit4]   2> 2479391 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/data
   [junit4]   2> 2479391 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2479391 WARN  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2479391 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/data/index
   [junit4]   2> 2479392 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@634898c7
   [junit4]   2> 2479392 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@332663bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d5e2787),segFN=segments_1,generation=1}
   [junit4]   2> 2479392 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2479393 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/data/
   [junit4]   2> 2479393 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2479397 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2479397 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2479397 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2479397 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2479397 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2479398 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2479398 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2479398 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2479398 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2479401 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2479402 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2479403 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2479404 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2479408 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/get,/admin/logging,/admin/mbeans,/admin/properties,/update/json,/admin/plugins,/update,/schema,/admin/file,/config,/replication,/admin/system,/admin/threads,/admin/luke,/admin/segments,/admin/ping,/update/json/docs,/update/csv
   [junit4]   2> 2479409 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2479409 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2479409 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2479410 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2479410 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2479411 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@794ae123
   [junit4]   2> 2479411 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@332663bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d5e2787),segFN=segments_1,generation=1}
   [junit4]   2> 2479411 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2479411 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3c37112b[collection1] main
   [junit4]   2> 2479411 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2479412 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2479412 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2479413 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2479413 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2479413 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2479413 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2479413 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2479414 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2479414 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2479414 INFO  (searcherExecutor-3558-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3c37112b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2479414 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2479414 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2479414 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2479414 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513367758128021504
   [junit4]   2> 2479417 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1513367758128021504
   [junit4]   2> 2479417 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2479417 INFO  (coreLoadExecutor-3557-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2479417 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:38471/j/ki collection:control_collection shard:shard1
   [junit4]   2> 2479418 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2479422 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94585484756254726-core_node1-n_0000000000
   [junit4]   2> 2479422 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2479424 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2479424 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2479424 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38471/j/ki/collection1/
   [junit4]   2> 2479424 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2479424 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:38471/j/ki/collection1/ has no replicas
   [junit4]   2> 2479424 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38471/j/ki/collection1/ shard1
   [junit4]   2> 2479425 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2479425 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2479427 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2479430 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:38471/j/ki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2479533 INFO  (zkCallback-1299-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2479547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2479548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2479549 INFO  (zkCallback-1301-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@176b349e name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2479549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2479550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2479550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2479552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2479552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2479552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2479552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2479554 INFO  (zkCallback-1302-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2e4095e4 name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2479554 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2479554 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2479558 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 2479559 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2479581 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:38471/j/ki/collection1/ and leader is http://127.0.0.1:38471/j/ki/collection1/
   [junit4]   2> 2479581 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38471/j/ki
   [junit4]   2> 2479581 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2479581 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2479581 INFO  (coreZkRegister-3551-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:38471_j%2Fki r:core_node1) [n:127.0.0.1:38471_j%2Fki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2479583 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:38471_j%2Fki",
   [junit4]   2>   "base_url":"http://127.0.0.1:38471/j/ki"} current state version: 2
   [junit4]   2> 2479583 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:38471_j%2Fki",
   [junit4]   2>   "base_url":"http://127.0.0.1:38471/j/ki"}
   [junit4]   2> 2479584 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2479688 INFO  (zkCallback-1301-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2479688 INFO  (zkCallback-1299-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2479826 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/cores/collection1
   [junit4]   2> 2479826 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001
   [junit4]   2> 2479827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2479829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a145065{/j/ki,null,AVAILABLE}
   [junit4]   2> 2479829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.ServerConnector Started ServerConnector@53684fd6{HTTP/1.1}{127.0.0.1:54260}
   [junit4]   2> 2479829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.e.j.s.Server Started @2482830ms
   [junit4]   2> 2479829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=54260, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/cores, hostContext=/j/ki, solrconfig=solrconfig.xml}
   [junit4]   2> 2479830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2479830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/'
   [junit4]   2> 2479845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2479846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2479847 INFO  (zkCallback-1303-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e44ce03 name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2479847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2479847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2479848 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2479849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/solr.xml
   [junit4]   2> 2479862 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/cores
   [junit4]   2> 2479862 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoreContainer New CoreContainer 1646307976
   [junit4]   2> 2479862 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/]
   [junit4]   2> 2479863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/lib
   [junit4]   2> 2479863 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/lib).
   [junit4]   2> 2479870 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2479872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2479872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2479872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2479872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45835/solr
   [junit4]   2> 2479872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2479872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2479873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2479875 INFO  (zkCallback-1305-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76f5f460 name:ZooKeeperConnection Watcher:127.0.0.1:45835 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2479875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2479876 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2479877 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2479879 INFO  (zkCallback-1306-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f557d62 name:ZooKeeperConnection Watcher:127.0.0.1:45835/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2479879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2479885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2480891 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54260_j%2Fki
   [junit4]   2> 2480892 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54260_j%2Fki
   [junit4]   2> 2480893 INFO  (zkCallback-1299-thread-1-processing-n:127.0.0.1:38471_j%2Fki) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2480893 INFO  (zkCallback-1301-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2480893 INFO  (zkCallback-1306-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2480895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2480895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94585484756254731-127.0.0.1:54260_j%2Fki-n_0000000001
   [junit4]   2> 2480896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000 to know if I could be the leader
   [junit4]   2> 2480918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2480918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2480919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/cores
   [junit4]   2> 2480920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, config=solrconfig.xml, name=collection1, schema=schema.xml, loadOnStartup=true, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/cores/collection1/, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/cores/collection1, coreNodeName=, dataDir=data/, configSetProperties=configsetprops.json}
   [junit4]   2> 2480920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49DB32BD14223847-001/shard-1-001/cores/collection1/
   [junit4]   2> 2480920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2480921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2480921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49DB32BD14223847]) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2480922 INFO  (coreLoadExecutor-3568-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2480922 INFO  (coreLoadExecutor-3568-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2480924 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:54260_j%2Fki",
   [junit4]   2>   "base_url":"http://127.0.0.1:54260/j/ki"} current state version: 3
   [junit4]   2> 2480924 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:54260_j%2Fki",
   [junit4]   2>   "base_url":"http://127.0.0.1:54260/j/ki"}
   [junit4]   2> 2480924 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 2480924 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 2480925 INFO  (coreLoadExecutor-3568-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2480925 INFO  (coreLoadExecutor-3568-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 2480925 INFO  (coreLoadExecutor-3568-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 2480926 INFO  (coreLoadExecutor-3568-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2481027 INFO  (OverseerStateUpdate-94585484756254726-127.0.0.1:38471_j%2Fki-n_0000000000) [n:127.0.0.1:38471_j%2Fki    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 2481028 INFO  (zkCallback-1306-thread-1-processing-n:127.0.0.1:54260_j%2Fki) [n:127.0.0.1:54260_j%2Fki    ] o.a.s.c.c.ZkStateReader A cl

[...truncated too long message...]

va.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
   [junit4]   2>         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:78)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
   [junit4]   2>         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
   [junit4]   2>         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
   [junit4]   2>         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>   31) Thread[id=54630, name=org.eclipse.jetty.server.session.HashSessionManager@5e9e833bTimer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>   32) Thread[id=54570, name=Scheduler-1455991037, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>   33) Thread[id=54598, name=Scheduler-671610382, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>   34) Thread[id=54586, name=searcherExecutor-2463-thread-1, state=WAITING, group=TGRP-DistributedVersionInfoTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {multiDefault=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), _version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_BH, timezone=America/Cancun
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=63,free=257999672,total=418906112
   [junit4]   2> NOTE: All tests run in this JVM: [TestCloudPivotFacet, JSONWriterTest, TestLuceneMatchVersion, TestCrossCoreJoin, TestArbitraryIndexDir, TestManagedSchema, TestSolrCLIRunExample, BigEndianAscendingWordDeserializerTest, CollectionTooManyReplicasTest, TestMaxScoreQueryParser, DistributedSpellCheckComponentTest, BlockDirectoryTest, DateFieldTest, CurrencyFieldOpenExchangeTest, HdfsChaosMonkeySafeLeaderTest, SharedFSAutoReplicaFailoverUtilsTest, TestRemoteStreaming, DeleteShardTest, TestMiniSolrCloudClusterSSL, TestCloudInspectUtil, TestSolr4Spatial, TestLMDirichletSimilarityFactory, ConcurrentDeleteAndCreateCollectionTest, SuggestComponentTest, TestDistributedSearch, AnalysisErrorHandlingTest, SOLR749Test, TestScoreJoinQPNoScore, TestLazyCores, TestRestoreCore, DefaultValueUpdateProcessorTest, DeleteInactiveReplicaTest, UUIDUpdateProcessorFallbackTest, StandardRequestHandlerTest, ResourceLoaderTest, FileUtilsTest, TestBadConfig, ExplicitHLLTest, TestOrdValues, DateMathParserTest, SyncSliceTest, TestConfigReload, EchoParamsTest, SolrXmlInZkTest, TestSolrXml, QueryElevationComponentTest, TestTrackingShardHandlerFactory, TestInfoStreamLogging, RecoveryAfterSoftCommitTest, DocumentAnalysisRequestHandlerTest, DistributedVersionInfoTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DistributedVersionInfoTest -Dtests.seed=49DB32BD14223847 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_BH -Dtests.timezone=America/Cancun -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | DistributedVersionInfoTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([49DB32BD14223847]:0)
   [junit4] Completed [501/539] on J0 in 7223.69s, 1 test, 2 errors <<< FAILURES!

[...truncated 130 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 1 suite-level error, 3 errors, 140 ignored (17 assumptions) [seed: 49DB32BD14223847]

Total time: 275 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1099699788 bytes
Compression is 0.0%
Took 2 min 54 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 966 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/966/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#3): CloudJettyRunner [url=http://127.0.0.1:54025/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#3): CloudJettyRunner [url=http://127.0.0.1:54025/collection1]
	at __randomizedtesting.SeedInfo.seed([8FE483CDADEEB3E6:7B0BC170312DE1E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.hdfs.StressHdfsTest.test

Error Message:
Error from server at http://127.0.0.1:49714/_s/kf: Could not find collection : delete_data_dir

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:49714/_s/kf: Could not find collection : delete_data_dir
	at __randomizedtesting.SeedInfo.seed([8FE483CDADEEB3E6:7B0BC170312DE1E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:206)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:56506: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /collections/acollectionafterbaddelete/state.json

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56506: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /collections/acollectionafterbaddelete/state.json
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=5083, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5083, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50804: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([8FE483CDADEEB3E6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10588 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/init-core-data-001
   [junit4]   2> 496159 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 496159 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 496758 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 497443 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 497611 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 497621 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 497701 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 497751 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_58218_hdfs____.vy6gow/webapp
   [junit4]   2> 497938 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 498921 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58218
   [junit4]   2> 499682 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 499689 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 499701 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42810_datanode____.pqrl34/webapp
   [junit4]   2> 499826 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 500391 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42810
   [junit4]   2> 500488 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 500490 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 500505 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57962_datanode____.14ils6/webapp
   [junit4]   2> 500629 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 501158 INFO  (IPC Server handler 3 on 34658) [    ] BlockStateChange BLOCK* processReport: from storage DS-5cbaa104-cb4d-4814-bf2b-23dd97e42181 node DatanodeRegistration(127.0.0.1, datanodeUuid=319447a4-1ec4-4d9e-967b-0aa39b460cc6, infoPort=42810, ipcPort=36044, storageInfo=lv=-56;cid=testClusterID;nsid=345570926;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 501158 INFO  (IPC Server handler 3 on 34658) [    ] BlockStateChange BLOCK* processReport: from storage DS-5e9b20e0-3a2f-4cea-ac85-80e15a9d816a node DatanodeRegistration(127.0.0.1, datanodeUuid=319447a4-1ec4-4d9e-967b-0aa39b460cc6, infoPort=42810, ipcPort=36044, storageInfo=lv=-56;cid=testClusterID;nsid=345570926;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 501161 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8FE483CDADEEB3E6]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57962
   [junit4]   2> 501318 INFO  (IPC Server handler 7 on 34658) [    ] BlockStateChange BLOCK* processReport: from storage DS-d7f2268f-6847-452d-966b-fc7280dad77b node DatanodeRegistration(127.0.0.1, datanodeUuid=c91346f0-d3a0-4b4a-b3b1-11ba742b2d52, infoPort=57962, ipcPort=53297, storageInfo=lv=-56;cid=testClusterID;nsid=345570926;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 501319 INFO  (IPC Server handler 7 on 34658) [    ] BlockStateChange BLOCK* processReport: from storage DS-8f5dbab5-0634-4ef3-aa33-f2ee075de0a0 node DatanodeRegistration(127.0.0.1, datanodeUuid=c91346f0-d3a0-4b4a-b3b1-11ba742b2d52, infoPort=57962, ipcPort=53297, storageInfo=lv=-56;cid=testClusterID;nsid=345570926;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 501996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 501996 INFO  (Thread-1554) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 501996 INFO  (Thread-1554) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 502096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.ZkTestServer start zk server on port:52901
   [junit4]   2> 502096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 502098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 502106 INFO  (zkCallback-399-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d1fad34 name:ZooKeeperConnection Watcher:127.0.0.1:52901 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 502106 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502106 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 502106 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 502110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 502110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 502113 INFO  (zkCallback-400-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a4c1880 name:ZooKeeperConnection Watcher:127.0.0.1:52901/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 502113 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502113 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 502113 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 502115 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 502116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 502118 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 502119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 502119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 502122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 502122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 502124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 502124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 502126 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 502126 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 502127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 502128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 502129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 502129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 502131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 502131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 502133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 502133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 502135 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 502136 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 502137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 502137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 502138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 502139 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 502142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 502143 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 502144 INFO  (zkCallback-401-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49cfe892 name:ZooKeeperConnection Watcher:127.0.0.1:52901 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 502144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 502144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 502146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 502147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 502148 INFO  (zkCallback-402-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6da57c60 name:ZooKeeperConnection Watcher:127.0.0.1:52901/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 502148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 502149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 502149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 502152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 502152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 502154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 502155 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 502157 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 502157 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 502160 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 502160 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 502163 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 502163 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 502165 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 502165 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 502167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 502167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 502170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 502170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 502173 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 502173 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 502174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 502175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 502176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 502177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 502460 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores/collection1
   [junit4]   2> 502463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 502465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e2d1237{/,null,AVAILABLE}
   [junit4]   2> 502466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1106f090{HTTP/1.1}{127.0.0.1:36402}
   [junit4]   2> 502466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.e.j.s.Server Started @505441ms
   [junit4]   2> 502466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=36402, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores, hostContext=/, solr.data.dir=hdfs://localhost:34658/hdfs__localhost_34658__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001_tempDir-002_control_data}
   [junit4]   2> 502466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 502466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/'
   [junit4]   2> 502482 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 502483 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 502484 INFO  (zkCallback-403-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@199369b name:ZooKeeperConnection Watcher:127.0.0.1:52901/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 502484 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502485 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 502487 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 502487 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/solr.xml
   [junit4]   2> 502503 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores
   [junit4]   2> 502503 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.CoreContainer New CoreContainer 725863817
   [junit4]   2> 502503 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/]
   [junit4]   2> 502503 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/lib
   [junit4]   2> 502503 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/lib).
   [junit4]   2> 502512 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 502513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 502514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 502514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 502514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52901/solr
   [junit4]   2> 502514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 502514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 502515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 502517 INFO  (zkCallback-405-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18ca419d name:ZooKeeperConnection Watcher:127.0.0.1:52901 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 502518 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502518 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 502521 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 502523 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1df02cc6 name:ZooKeeperConnection Watcher:127.0.0.1:52901/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 502523 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502524 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 502528 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 502533 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 502535 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 502538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 502541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 502543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 502545 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 502546 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 502548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36402_
   [junit4]   2> 502548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36402_
   [junit4]   2> 502550 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 502551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 502553 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 502554 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94578706214682630-127.0.0.1:36402_-n_0000000000
   [junit4]   2> 502555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36402_
   [junit4]   2> 502555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 502557 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.Overseer Overseer (id=94578706214682630-127.0.0.1:36402_-n_0000000000) starting
   [junit4]   2> 502560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 502569 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 502570 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 502570 INFO  (OverseerCollectionConfigSetProcessor-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 502570 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 502578 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 502601 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 502601 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 502602 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores
   [junit4]   2> 502603 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, dataDir=data/, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores/collection1, coreNodeName=, loadOnStartup=true, config=solrconfig.xml, name=collection1, schema=schema.xml, transient=false, shard=}
   [junit4]   2> 502603 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores/collection1/
   [junit4]   2> 502603 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 502604 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 502604 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[8FE483CDADEEB3E6]) [n:127.0.0.1:36402_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 502605 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 502605 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 502607 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:36402",
   [junit4]   2>   "node_name":"127.0.0.1:36402_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 502607 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:36402",
   [junit4]   2>   "node_name":"127.0.0.1:36402_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 502608 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 502608 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 502608 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 502610 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 502611 INFO  (zkCallback-406-thread-2-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 503608 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 503609 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 503609 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 503609 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 503610 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 503610 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores/collection1/'
   [junit4]   2> 503625 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 503633 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 503647 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 503682 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 503683 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 503705 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 504685 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 504694 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 504696 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 504707 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 504712 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 504715 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 504716 WARN  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 504716 WARN  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 504716 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 504716 WARN  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 504717 WARN  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 504717 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 504717 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 504718 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 504726 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:34658/solr_hdfs_home
   [junit4]   2> 504726 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 504727 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8FE483CDADEEB3E6-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 504727 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@153ad7f4
   [junit4]   2> 504727 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 504787 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 504788 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 504793 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 504813 WARN  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:34658/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 504813 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 504832 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 504832 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 504832 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=12, maxMergedSegmentMB=61.2490234375, floorSegmentMB=1.4013671875, forceMergeDeletesPctAllowed=12.150228359648278, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 505118 INFO  (IPC Server handler 4 on 34658) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34517 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5e9b20e0-3a2f-4cea-ac85-80e15a9d816a:NORMAL:127.0.0.1:34517|RBW], ReplicaUnderConstruction[[DISK]DS-8f5dbab5-0634-4ef3-aa33-f2ee075de0a0:NORMAL:127.0.0.1:37613|RBW]]} size 71
   [junit4]   2> 505119 INFO  (IPC Server handler 0 on 34658) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:37613 is added to blk_1073741825_1001 size 71
   [junit4]   2> 505514 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6300590 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d5898ed; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 505514 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 505518 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:34658/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 505524 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 505525 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 505525 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 505525 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 505525 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 505525 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 505525 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 505526 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 505526 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 505528 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 505530 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 505531 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 505532 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 505536 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,standard,/admin/segments,/update,/admin/properties,/admin/file,/admin/mbeans,/admin/system,/admin/logging,/replication,/admin/threads,/get,/admin/plugins,/update/json/docs,/update/csv,/schema,/config,/admin/ping,/admin/luke
   [junit4]   2> 505538 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 505538 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 505538 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 505568 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 505568 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 505571 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 505641 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6300590 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d5898ed; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 505641 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 505641 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@59c8b405[collection1] main
   [junit4]   2> 505642 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 505642 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 505642 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 505643 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 505643 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 505643 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 505644 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 505644 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 505644 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 505644 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 505645 INFO  (searcherExecutor-1444-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59c8b405[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 505652 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 505652 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 505652 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 505652 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513259303266418688
   [junit4]   2> 505653 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1513259303266418688
   [junit4]   2> 505653 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 505653 INFO  (coreLoadExecutor-1443-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 505654 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:36402 collection:control_collection shard:shard1
   [junit4]   2> 505655 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 505659 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94578706214682630-core_node1-n_0000000000
   [junit4]   2> 505659 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 505661 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 505661 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 505661 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36402/collection1/
   [junit4]   2> 505661 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 505661 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36402/collection1/ has no replicas
   [junit4]   2> 505661 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36402/collection1/ shard1
   [junit4]   2> 505661 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 505662 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 505663 INFO  (zkCallback-406-thread-2-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 505663 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 505667 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 505672 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36402",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 505775 INFO  (zkCallback-406-thread-2-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 505775 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:36402_) [n:127.0.0.1:36402_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 505822 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:36402/collection1/ and leader is http://127.0.0.1:36402/collection1/
   [junit4]   2> 505822 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36402
   [junit4]   2> 505823 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 505823 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 505823 INFO  (coreZkRegister-1437-thread-1-processing-n:127.0.0.1:36402_ c:control_collection s:shard1 x:collection1 r:core_node1) [n:127.0.0.1:36402_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 505825 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:36402",
   [junit4]   2>   "node_name":"127.0.0.1:36402_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 505826 INFO  (OverseerStateUpdate-94578706214682630-127.0.0.1:36402_-n_0000000000) [n:127.0.0.1:36402_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:36402",
   [junit4]   2>   "node_name":"127.0.0.1:36402_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_na

[...truncated too long message...]

icsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.StressHdfsTest.teardownClass(StressHdfsTest.java:74)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4941055 INFO  (SUITE-StressHdfsTest-seed#[8FE483CDADEEB3E6]-worker) [n:127.0.0.1:39853__s%2Fkf c:collection1 s:shard1 r:core_node7 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_8FE483CDADEEB3E6-001
   [junit4]   2> Sep 25, 2015 2:02:47 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {txt_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), multiDefault=PostingsFormat(name=Direct), id=Lucene50(blocksize=128), _version_=PostingsFormat(name=MockRandom), intDefault=PostingsFormat(name=MockRandom), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=Lucene50(blocksize=128), timestamp=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), range_facet_i_dv=PostingsFormat(name=Direct)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=de_LU, timezone=Asia/Singapore
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=246153544,total=529006592
   [junit4]   2> NOTE: All tests run in this JVM: [CurrencyFieldOpenExchangeTest, DistributedVersionInfoTest, TestTrackingShardHandlerFactory, ConcurrentDeleteAndCreateCollectionTest, InfoHandlerTest, QueryElevationComponentTest, TestComplexPhraseQParserPlugin, CursorPagingTest, SignatureUpdateProcessorFactoryTest, TestConfig, TestDocSet, TestPseudoReturnFields, CoreMergeIndexesAdminHandlerTest, ParsingFieldUpdateProcessorsTest, DistributedIntervalFacetingTest, TestDocBasedVersionConstraints, TestCSVLoader, TestSolr4Spatial, TestSolr4Spatial2, TestDocumentBuilder, BigEndianAscendingWordDeserializerTest, TestRandomFaceting, TestAnalyzedSuggestions, TestAuthenticationFramework, CustomCollectionTest, TestClusterStateMutator, TestRemoteStreaming, PolyFieldTest, TestCrossCoreJoin, TestManagedStopFilterFactory, TestCloudPivotFacet, DefaultValueUpdateProcessorTest, TestFoldingMultitermQuery, DistributedQueryElevationComponentTest, EchoParamsTest, OverseerStatusTest, DateMathParserTest, SuggestComponentTest, JavabinLoaderTest, TestRangeQuery, MultiThreadedOCPTest, TestReqParamsAPI, TestConfigReload, TestRandomRequestDistribution, DistributedMLTComponentTest, TestChildDocTransformer, AnalysisAfterCoreReloadTest, HdfsNNFailoverTest, AlternateDirectoryTest, FileBasedSpellCheckerTest, TestReplicationHandler, TestInfoStreamLogging, IndexSchemaTest, SolrRequestParserTest, PrimUtilsTest, SolrCoreTest, RequiredFieldsTest, CopyFieldTest, TestMiniSolrCloudClusterSSL, RemoteQueryErrorTest, UUIDUpdateProcessorFallbackTest, TestStressLucene, TestSolrXml, TestManagedSchema, TriLevelCompositeIdRoutingTest, FileUtilsTest, TimeZoneUtilsTest, SpellingQueryConverterTest, TestScoreJoinQPScore, TestLRUStatsCache, FullHLLTest, TestInitQParser, OutputWriterTest, LeaderElectionIntegrationTest, TestCollationFieldDocValues, SynonymTokenizerTest, TestManagedSchemaDynamicFieldResource, OpenExchangeRatesOrgProviderTest, SharedFSAutoReplicaFailoverUtilsTest, SOLR749Test, AliasIntegrationTest, TestMaxScoreQueryParser, TestLMDirichletSimilarityFactory, TestDistribDocBasedVersion, ResponseLogComponentTest, BasicDistributedZkTest, DistributedFacetPivotLargeTest, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest, SimpleCollectionCreateDeleteTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCollectionAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestExclusionRuleCollectionAccess, TestLeaderInitiatedRecoveryThread, TestRebalanceLeaders, TestShortCircuitedRequests, HdfsBasicDistributedZk2Test, HdfsRecoverLeaseTest, HdfsThreadLeakTest, StressHdfsTest]
   [junit4] Completed [449/539] on J0 in 194.18s, 1 test, 1 error <<< FAILURES!

[...truncated 297 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 3 errors, 1 failure, 29 ignored (17 assumptions) [seed: 8FE483CDADEEB3E6]

Total time: 203 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1216225047 bytes
Compression is 0.0%
Took 3 min 12 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 965 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/965/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15224, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15224, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46029: Could not find collection : awholynewstresscollection_collection1_5
	at __randomizedtesting.SeedInfo.seed([909429C3CC447CA5]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=105154, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=105154, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57765/gn: Could not find collection : awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([909429C3CC447CA5]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10683 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/init-core-data-001
   [junit4]   2> 1744228 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[909429C3CC447CA5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1744228 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[909429C3CC447CA5]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1744232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1744233 INFO  (Thread-8564) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1744233 INFO  (Thread-8564) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1744333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.ZkTestServer start zk server on port:40896
   [junit4]   2> 1744333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1744333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1744341 INFO  (zkCallback-1401-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69a1dd11 name:ZooKeeperConnection Watcher:127.0.0.1:40896 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1744341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1744341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1744341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1744347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1744347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1744349 INFO  (zkCallback-1402-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43f40b48 name:ZooKeeperConnection Watcher:127.0.0.1:40896/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1744349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1744349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1744349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1744351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1744355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1744358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1744361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1744361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1744368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1744368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1744371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1744371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1744372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1744373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1744375 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1744376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1744379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1744379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1744385 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1744385 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1744391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1744391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1744400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1744401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1744412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1744413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1744414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1744415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1744779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1
   [junit4]   2> 1744780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1744782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55e1347e{/,null,AVAILABLE}
   [junit4]   2> 1744784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2b86d93d{HTTP/1.1}{127.0.0.1:46029}
   [junit4]   2> 1744786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.Server Started @1747913ms
   [junit4]   2> 1744786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=46029, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/tempDir-001/control/data}
   [junit4]   2> 1744787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1744787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/'
   [junit4]   2> 1744806 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1744829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1744831 INFO  (zkCallback-1403-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@109767fa name:ZooKeeperConnection Watcher:127.0.0.1:40896/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1744831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1744831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1744832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1744832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/solr.xml
   [junit4]   2> 1744853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores
   [junit4]   2> 1744853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoreContainer New CoreContainer 1482312221
   [junit4]   2> 1744853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/]
   [junit4]   2> 1744853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/lib
   [junit4]   2> 1744853 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/lib).
   [junit4]   2> 1744867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1744868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1744869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1744869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1744869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40896/solr
   [junit4]   2> 1744869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1744869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1744870 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1744871 INFO  (zkCallback-1405-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@339530d9 name:ZooKeeperConnection Watcher:127.0.0.1:40896 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1744871 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1744871 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1744873 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14fff28439a0003, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1744884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1744889 INFO  (zkCallback-1406-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bd34a96 name:ZooKeeperConnection Watcher:127.0.0.1:40896/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1744889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1744891 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1744894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1744899 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1744905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1744909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1744912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1744913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1744915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1744916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1744917 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46029_
   [junit4]   2> 1744918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46029_
   [junit4]   2> 1744922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1744930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1744935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1744936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94574665596010500-127.0.0.1:46029_-n_0000000000
   [junit4]   2> 1744936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46029_
   [junit4]   2> 1744936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1744938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer Overseer (id=94574665596010500-127.0.0.1:46029_-n_0000000000) starting
   [junit4]   2> 1744939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1744947 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1744947 INFO  (OverseerCollectionConfigSetProcessor-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1744948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1744949 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1744974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1744975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1744976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores
   [junit4]   2> 1744977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1, config=solrconfig.xml, coreNodeName=, collection=control_collection, loadOnStartup=true, transient=false, configSetProperties=configsetprops.json, name=collection1}
   [junit4]   2> 1744977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/
   [junit4]   2> 1744977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1744979 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1744979 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1744981 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1744984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1744984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:46029_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1744985 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:46029",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46029_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 1744985 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:46029",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46029_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null}
   [junit4]   2> 1744986 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1744986 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1744998 INFO  (zkCallback-1406-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1745982 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1745982 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1745983 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1745983 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1745984 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1745984 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/'
   [junit4]   2> 1746000 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1746014 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1746028 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1746056 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1746057 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1746064 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1746335 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1746346 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1746347 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1746364 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1746367 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1746372 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1746373 WARN  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1746374 WARN  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1746374 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1746375 WARN  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1746375 WARN  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1746376 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1746376 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1746376 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1746377 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1746377 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f125e52
   [junit4]   2> 1746377 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/data
   [junit4]   2> 1746377 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1746377 WARN  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1746378 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/data/index
   [junit4]   2> 1746378 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1746379 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5934751a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f42bb7; maxCacheMB=0.7548828125 maxMergeSizeMB=0.40625)),segFN=segments_1,generation=1}
   [junit4]   2> 1746379 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1746379 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/data/
   [junit4]   2> 1746380 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1746383 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1746384 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1746384 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1746384 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1746384 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1746384 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1746384 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1746384 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1746385 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1746388 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1746389 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1746391 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1746392 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1746395 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,/admin/plugins,/admin/file,/admin/properties,/admin/mbeans,/replication,/schema,/admin/ping,standard,/update/csv,/update,/admin/threads,/admin/system,/update/json,/config,/admin/luke,/admin/segments,/get,/admin/logging
   [junit4]   2> 1746396 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1746397 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1746397 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1746398 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1746398 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1746398 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=4, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1746449 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5934751a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f42bb7; maxCacheMB=0.7548828125 maxMergeSizeMB=0.40625)),segFN=segments_1,generation=1}
   [junit4]   2> 1746449 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1746449 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@104a1259[collection1] main
   [junit4]   2> 1746449 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1746450 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1746450 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1746451 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1746451 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1746451 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1746451 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1746451 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1746451 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1746451 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1746452 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1746452 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1746452 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1746452 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513194651861909504
   [junit4]   2> 1746454 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1513194651861909504
   [junit4]   2> 1746454 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1746455 INFO  (searcherExecutor-3499-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@104a1259[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1746456 INFO  (coreLoadExecutor-3498-thread-1-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1746456 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46029 collection:control_collection shard:shard1
   [junit4]   2> 1746467 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1746473 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94574665596010500-core_node1-n_0000000000
   [junit4]   2> 1746474 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1746475 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1746475 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1746476 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46029/collection1/
   [junit4]   2> 1746476 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1746476 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46029/collection1/ has no replicas
   [junit4]   2> 1746476 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46029/collection1/ shard1
   [junit4]   2> 1746476 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1746477 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1746479 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1746481 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46029",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1746486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1746487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1746488 INFO  (zkCallback-1408-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78c872cb name:ZooKeeperConnection Watcher:127.0.0.1:40896/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1746488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1746488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1746488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1746490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1746584 INFO  (zkCallback-1408-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1746584 INFO  (zkCallback-1406-thread-2-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1746632 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46029/collection1/ and leader is http://127.0.0.1:46029/collection1/
   [junit4]   2> 1746632 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46029
   [junit4]   2> 1746632 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1746632 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1746632 INFO  (coreZkRegister-3492-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:46029_ s:shard1 r:core_node1) [n:127.0.0.1:46029_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1746635 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"http://127.0.0.1:46029",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:46029_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1"} current state version: 2
   [junit4]   2> 1746635 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"http://127.0.0.1:46029",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:46029_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 1746738 INFO  (zkCallback-1408-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1746738 INFO  (zkCallback-1406-thread-2-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1746884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1
   [junit4]   2> 1746885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001
   [junit4]   2> 1746885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1746889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@db7ceee{/,null,AVAILABLE}
   [junit4]   2> 1746890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6bada98a{HTTP/1.1}{127.0.0.1:41751}
   [junit4]   2> 1746890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.e.j.s.Server Started @1750017ms
   [junit4]   2> 1746890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores, hostPort=41751}
   [junit4]   2> 1746890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1746890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/'
   [junit4]   2> 1746919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1746920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1746921 INFO  (zkCallback-1409-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39fc1150 name:ZooKeeperConnection Watcher:127.0.0.1:40896/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1746921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1746921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1746922 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14fff28439a0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1746923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1746923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/solr.xml
   [junit4]   2> 1746943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores
   [junit4]   2> 1746944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoreContainer New CoreContainer 156237730
   [junit4]   2> 1746944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/]
   [junit4]   2> 1746944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/lib
   [junit4]   2> 1746944 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/lib).
   [junit4]   2> 1746953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1746955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1746955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1746955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1746955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40896/solr
   [junit4]   2> 1746955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1746956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1746958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1746959 INFO  (zkCallback-1411-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6aeb5f60 name:ZooKeeperConnection Watcher:127.0.0.1:40896 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1746960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1746960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1746962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1746963 INFO  (zkCallback-1412-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@442591fd name:ZooKeeperConnection Watcher:127.0.0.1:40896/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1746963 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1746974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1747979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41751_
   [junit4]   2> 1747980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41751_
   [junit4]   2> 1747981 INFO  (zkCallback-1406-thread-2-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1747981 INFO  (zkCallback-1412-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1747981 INFO  (zkCallback-1408-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1747984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1747985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94574665596010504-127.0.0.1:41751_-n_0000000001
   [junit4]   2> 1747985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94574665596010500-127.0.0.1:46029_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1748011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1748011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1748012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores
   [junit4]   2> 1748014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/, shard=, config=solrconfig.xml, loadOnStartup=true, transient=false, coreNodeName=, configSetProperties=configsetprops.json, name=collection1, collection=collection1, dataDir=data/, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1}
   [junit4]   2> 1748014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/
   [junit4]   2> 1748014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1748015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1748015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [n:127.0.0.1:41751_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1748017 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1748017 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1748022 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:41751",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:41751_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null} current state version: 3
   [junit4]   2> 1748022 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1748023 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:41751",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:41751_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null}
   [junit4]   2> 1748023 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1748023 INFO  (OverseerStateUpdate-94574665596010500-127.0.0.1:46029_-n_0000000000) [n:127.0.0.1:46029_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1748130 INFO  (zkCallback-1408-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1748131 INFO  (zkCallback-1412-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1748131 INFO  (zkCallback-1406-thread-2-processing-n:127.0.0.1:46029_) [n:127.0.0.1:46029_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1749025 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1749025 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1749025 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1749025 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1749026 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1749026 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1749044 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1749050 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1749083 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1749114 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1749115 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1749121 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1749391 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1749401 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1749402 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1749418 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1749421 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1749429 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1749430 WARN  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1749430 WARN  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1749430 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1749431 WARN  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1749431 WARN  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1749432 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1749432 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1749432 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1749432 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1749432 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f125e52
   [junit4]   2> 1749433 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1749433 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1749433 WARN  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1749433 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1749433 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1749434 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@36f35437 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@211e0310; maxCacheMB=0.7548828125 maxMergeSizeMB=0.40625)),segFN=segments_1,generation=1}
   [junit4]   2> 1749434 INFO  (coreLoadExecutor-3509-thread-1-processing-n:127.0.0.1:41751_) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1749434 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/data/
   [junit4]   2> 1749435 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41751_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/shard-1-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1749439 INFO  (c

[...truncated too long message...]

8)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4878723 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[909429C3CC447CA5]-worker) [n:127.0.0.1:32846_gn c:addReplicaColl s:shard2 r:core_node2 x:addReplicaColl_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 24, 2015 8:33:27 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {_version_=PostingsFormat(name=Memory doPackFST= true), timestamp=FSTOrd50, range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), other_tl1=PostingsFormat(name=Memory doPackFST= true), multiDefault=FSTOrd50, a_t=FSTOrd50, text=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=FSTOrd50, range_facet_i_dv=FSTOrd50, intDefault=PostingsFormat(name=Memory doPackFST= true), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_i1=FSTOrd50}, docValues:{timestamp=DocValuesFormat(name=Lucene50), range_facet_l_dv=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=zh_TW, timezone=Asia/Tashkent
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=50410200,total=526385152
   [junit4]   2> NOTE: All tests run in this JVM: [DisMaxRequestHandlerTest, TestAnalyzedSuggestions, TestLMDirichletSimilarityFactory, DeleteInactiveReplicaTest, TestSolrJ, MultiThreadedOCPTest, IndexSchemaRuntimeFieldTest, HighlighterMaxOffsetTest, TestBinaryField, TestRandomDVFaceting, TestTrieFacet, DistributedFacetPivotWhiteBoxTest, TestConfig, StressHdfsTest, CacheHeaderTest, ExplicitHLLTest, TestConfigSetProperties, TestFaceting, JsonLoaderTest, TestScoreJoinQPNoScore, TestDocBasedVersionConstraints, TestExpandComponent, QueryElevationComponentTest, TestHighlightDedupGrouping, AliasIntegrationTest, CoreMergeIndexesAdminHandlerTest, TestCSVLoader, TestClusterStateMutator, TestMacros, HdfsBasicDistributedZk2Test, TestCollectionAPI, JavabinLoaderTest, DistributedQueryElevationComponentTest, OverseerStatusTest, TestJoin, TestInfoStreamLogging, TestExactStatsCache, RecoveryAfterSoftCommitTest, TestRemoteStreaming, TestSolr4Spatial, TestFunctionQuery, PingRequestHandlerTest, BasicZkTest, TestMiniSolrCloudClusterSSL, CollectionsAPIAsyncDistributedZkTest, DeleteShardTest, TestReplicationHandler, InfoHandlerTest, TestOmitPositions, DateFieldTest, JSONWriterTest, SyncSliceTest, TestAtomicUpdateErrorCases, HdfsChaosMonkeySafeLeaderTest, TestBulkSchemaConcurrent, TestStressLucene, LeaderInitiatedRecoveryOnCommitTest, TestRealTimeGet, DistributedIntervalFacetingTest, IndexSchemaTest, PolyFieldTest, TestCryptoKeys, TestRandomFaceting, TestSerializedLuceneMatchVersion, TestSolrConfigHandlerConcurrent, OpenExchangeRatesOrgProviderTest, TestSchemaSimilarityResource, SharedFSAutoReplicaFailoverUtilsTest, TestReqParamsAPI, TestLRUStatsCache, TestSolrXml, PrimUtilsTest, TestSha256AuthenticationProvider, DistributedQueryComponentOptimizationTest, TestCloudPivotFacet, ExternalFileFieldSortTest, BlockCacheTest, TestFoldingMultitermQuery, FullHLLTest, TestManagedSchemaDynamicFieldResource, DistributedSpellCheckComponentTest, TestTrackingShardHandlerFactory, UUIDUpdateProcessorFallbackTest, TestCollationFieldDocValues, SpellingQueryConverterTest, QueryEqualityTest, XsltUpdateRequestHandlerTest, TestManagedStopFilterFactory, CursorPagingTest, LeaderElectionIntegrationTest, FullSolrCloudDistribCmdsTest, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, TestQuerySenderListener, TestSolrIndexConfig, ResponseLogComponentTest, BadComponentTest, TestStressRecovery, SynonymTokenizerTest, EchoParamsTest, TestDFRSimilarityFactory, TestDefaultSimilarityFactory, TimeZoneUtilsTest, TestFastOutputStream, URLClassifyProcessorTest, DateMathParserTest, DOMUtilTest, TestDocumentBuilder, SystemInfoHandlerTest, FileUtilsTest, DistributedMLTComponentTest, TestRTGBase, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestTolerantSearch, AsyncCallRequestStatusResponseTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionStateFormat2Test, ConcurrentDeleteAndCreateCollectionTest, DistribJoinFromCollectionTest, HttpPartitionTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudInspectUtil, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestExclusionRuleCollectionAccess, TestLeaderInitiatedRecoveryThread, TestRebalanceLeaders, TestSolrCloudWithKerberosAlt, VMParamsZkACLAndCredentialsProvidersTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [539/539] on J0 in 1172.41s, 1 test, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites (7 ignored), 2128 tests, 2 errors, 29 ignored (17 assumptions) [seed: 909429C3CC447CA5]

Total time: 205 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1317787964 bytes
Compression is 0.0%
Took 3 min 33 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 964 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/964/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=110251, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=110251, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37705: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([225CBFC8AC184BB8]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10971 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150923_002555_558.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid26872.hprof ...
   [junit4] Heap dump file created [682232877 bytes in 8.791 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150923_002555_558.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] 	at java.nio.CharBuffer.wrap(CharBuffer.java:369)
   [junit4] 	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:265)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135)
   [junit4] 	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220)
   [junit4] 	at java.io.Writer.write(Writer.java:157)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.stream.JsonWriter.string(JsonWriter.java:561)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.stream.JsonWriter.value(JsonWriter.java:419)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:101)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:83)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:457)
   [junit4] 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
   [junit4] 	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
   [junit4] 	at java.io.PrintStream.flush(PrintStream.java:338)
   [junit4] 	at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
   [junit4] 	at java.io.PrintStream.write(PrintStream.java:482)
   [junit4] 	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4] 	at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4] 	at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4] 	at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4] 	at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4] 	at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
   [junit4] 	at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
   [junit4] 	at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
   [junit4] 	at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
   [junit4] 	at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
   [junit4] 	at org.apache.log4j.Category.callAppenders(Category.java:206)
   [junit4] 	at org.apache.log4j.Category.forcedLog(Category.java:391)
   [junit4] 	at org.apache.log4j.Category.log(Category.java:856)
   [junit4] 	at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:497)
   [junit4] 	at org.apache.solr.common.SolrException.log(SolrException.java:139)
   [junit4] 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:169)
   [junit4] <<< JVM J2: EOF ----

[...truncated 684 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/init-core-data-001
   [junit4]   2> 3958846 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 3958846 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3958896 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3958906 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3958910 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3958931 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_47227_hdfs____.2l8gci/webapp
   [junit4]   2> 3959066 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3959409 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47227
   [junit4]   2> 3959471 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3959473 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3959485 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36096_datanode____.m7axan/webapp
   [junit4]   2> 3959598 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3959950 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36096
   [junit4]   2> 3959999 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3960001 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3960017 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34538_datanode____hqsb0g/webapp
   [junit4]   2> 3960069 INFO  (IPC Server handler 3 on 58563) [    ] BlockStateChange BLOCK* processReport: from storage DS-706fe534-26bd-4e42-92be-2b735d52eff5 node DatanodeRegistration(127.0.0.1, datanodeUuid=fd7b5557-b3e6-4077-a076-e1c6cda4cd0a, infoPort=36096, ipcPort=51304, storageInfo=lv=-56;cid=testClusterID;nsid=1823722244;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3960069 INFO  (IPC Server handler 3 on 58563) [    ] BlockStateChange BLOCK* processReport: from storage DS-bd3fe8e3-ae10-43c2-bd15-4478627b88f0 node DatanodeRegistration(127.0.0.1, datanodeUuid=fd7b5557-b3e6-4077-a076-e1c6cda4cd0a, infoPort=36096, ipcPort=51304, storageInfo=lv=-56;cid=testClusterID;nsid=1823722244;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3960145 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3960450 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[225CBFC8AC184BB8]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34538
   [junit4]   2> 3960521 INFO  (IPC Server handler 4 on 58563) [    ] BlockStateChange BLOCK* processReport: from storage DS-0c7a26ab-25b3-4966-9367-83b09545271b node DatanodeRegistration(127.0.0.1, datanodeUuid=b7d1adab-8f05-421f-99c6-12ca0cc1c005, infoPort=34538, ipcPort=41200, storageInfo=lv=-56;cid=testClusterID;nsid=1823722244;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3960522 INFO  (IPC Server handler 4 on 58563) [    ] BlockStateChange BLOCK* processReport: from storage DS-d99be8ff-721c-4624-b604-2a955f3594ff node DatanodeRegistration(127.0.0.1, datanodeUuid=b7d1adab-8f05-421f-99c6-12ca0cc1c005, infoPort=34538, ipcPort=41200, storageInfo=lv=-56;cid=testClusterID;nsid=1823722244;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3960609 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3960610 INFO  (Thread-98483) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3960610 INFO  (Thread-98483) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3960710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.ZkTestServer start zk server on port:52518
   [junit4]   2> 3960710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3960710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3960713 INFO  (zkCallback-2135-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1901d29d name:ZooKeeperConnection Watcher:127.0.0.1:52518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3960713 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3960714 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3960714 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3960716 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3960717 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3960718 INFO  (zkCallback-2136-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1b4e8ca6 name:ZooKeeperConnection Watcher:127.0.0.1:52518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3960718 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3960718 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3960718 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3960720 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3960721 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3960722 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3960724 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3960724 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3960726 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3960726 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3960728 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3960728 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3960729 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3960730 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3960731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3960731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3960733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3960733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3960734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3960734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3960736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3960736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3960738 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3960738 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3960739 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3960739 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3960741 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3960741 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3960743 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3960743 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3960744 INFO  (zkCallback-2137-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5fd77e48 name:ZooKeeperConnection Watcher:127.0.0.1:52518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3960744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3960745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3961033 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores/collection1
   [junit4]   2> 3961035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3961037 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c1d0005{/,null,AVAILABLE}
   [junit4]   2> 3961040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2999f19c{SSL-http/1.1}{127.0.0.1:52355}
   [junit4]   2> 3961040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.Server Started @3964126ms
   [junit4]   2> 3961040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=hdfs://localhost:58563/hdfs__localhost_58563__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001_tempDir-002_control_data, hostPort=52355, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores}
   [junit4]   2> 3961040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3961041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/'
   [junit4]   2> 3961056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3961057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3961058 INFO  (zkCallback-2138-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4cdb930e name:ZooKeeperConnection Watcher:127.0.0.1:52518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3961058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3961059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3961059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3961059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/solr.xml
   [junit4]   2> 3961073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores
   [junit4]   2> 3961073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoreContainer New CoreContainer 2048060521
   [junit4]   2> 3961073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/]
   [junit4]   2> 3961073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/lib
   [junit4]   2> 3961073 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/lib).
   [junit4]   2> 3961081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3961083 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3961085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3961085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3961085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52518/solr
   [junit4]   2> 3961085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3961085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3961085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3961087 INFO  (zkCallback-2140-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2cce52ac name:ZooKeeperConnection Watcher:127.0.0.1:52518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3961087 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3961087 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3961088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3961089 INFO  (zkCallback-2141-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@685cbeec name:ZooKeeperConnection Watcher:127.0.0.1:52518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3961089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3961090 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3961092 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3961095 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3961097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3961099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3961101 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3961102 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3961103 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3961104 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3961105 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52355_
   [junit4]   2> 3961105 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52355_
   [junit4]   2> 3961107 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3961108 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3961109 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3961110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94567551427280901-127.0.0.1:52355_-n_0000000000
   [junit4]   2> 3961110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52355_
   [junit4]   2> 3961110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3961111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer Overseer (id=94567551427280901-127.0.0.1:52355_-n_0000000000) starting
   [junit4]   2> 3961113 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3961118 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3961119 INFO  (OverseerCollectionConfigSetProcessor-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3961119 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3961119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3961152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3961152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3961153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores
   [junit4]   2> 3961154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, shard=, collection=control_collection, loadOnStartup=true, name=collection1, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores/collection1, schema=schema.xml, coreNodeName=, transient=false, dataDir=data/}
   [junit4]   2> 3961154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores/collection1/
   [junit4]   2> 3961154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3961155 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3961155 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:52355_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3961155 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3961155 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3961157 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:52355_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"https://127.0.0.1:52355",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 3961157 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3961157 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:52355_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"https://127.0.0.1:52355",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null}
   [junit4]   2> 3961158 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3961158 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3961159 INFO  (zkCallback-2141-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3962157 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3962158 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3962158 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3962158 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3962159 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3962159 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores/collection1/'
   [junit4]   2> 3962172 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3962178 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3962203 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3962231 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3962232 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3962239 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3962483 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3962490 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3962492 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3962501 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3962504 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3962507 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3962507 WARN  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3962508 WARN  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3962508 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3962508 WARN  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3962508 WARN  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3962509 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3962509 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3962510 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3962510 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58563/solr_hdfs_home
   [junit4]   2> 3962510 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3962510 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3962510 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@22b6b6e1
   [junit4]   2> 3962510 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3962526 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 3962526 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3962527 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3962539 WARN  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:58563/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3962539 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3962552 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 3962552 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3962553 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=43, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.26619935206227097]
   [junit4]   2> 3962567 INFO  (IPC Server handler 8 on 58563) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34393 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-706fe534-26bd-4e42-92be-2b735d52eff5:NORMAL:127.0.0.1:50565|RBW], ReplicaUnderConstruction[[DISK]DS-d99be8ff-721c-4624-b604-2a955f3594ff:NORMAL:127.0.0.1:34393|RBW]]} size 0
   [junit4]   2> 3962568 INFO  (IPC Server handler 9 on 58563) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50565 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-706fe534-26bd-4e42-92be-2b735d52eff5:NORMAL:127.0.0.1:50565|RBW], ReplicaUnderConstruction[[DISK]DS-d99be8ff-721c-4624-b604-2a955f3594ff:NORMAL:127.0.0.1:34393|RBW]]} size 0
   [junit4]   2> 3962570 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8f6f10d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@10a0a67a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3962570 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3962571 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:58563/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3962575 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3962575 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3962576 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3962576 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3962576 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3962576 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3962576 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3962576 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3962576 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3962579 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3962580 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3962581 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3962583 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3962586 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/config,/get,/update/json/docs,/admin/luke,/admin/ping,standard,/admin/threads,/update/json,/admin/file,/admin/mbeans,/admin/plugins,/admin/segments,/admin/system,/update/csv,/replication,/admin/logging,/admin/properties,/schema
   [junit4]   2> 3962588 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3962588 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3962588 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 3962598 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3962599 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3962600 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=0.7080078125, floorSegmentMB=1.283203125, forceMergeDeletesPctAllowed=2.576653482271891, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2542541807328468
   [junit4]   2> 3962605 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8f6f10d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@10a0a67a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3962605 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3962605 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@e500cf1[collection1] main
   [junit4]   2> 3962605 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3962606 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3962606 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3962606 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3962606 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3962606 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3962607 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3962607 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3962607 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3962607 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3962607 INFO  (searcherExecutor-7043-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@e500cf1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3962609 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3962610 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 3962610 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3962610 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1513080824932597760
   [junit4]   2> 3962610 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1513080824932597760
   [junit4]   2> 3962610 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3962610 INFO  (coreLoadExecutor-7042-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3962611 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:52355 collection:control_collection shard:shard1
   [junit4]   2> 3962611 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3962614 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94567551427280901-core_node1-n_0000000000
   [junit4]   2> 3962615 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3962616 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3962616 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3962616 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:52355/collection1/
   [junit4]   2> 3962617 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3962617 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:52355/collection1/ has no replicas
   [junit4]   2> 3962617 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3962617 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:52355/collection1/ shard1
   [junit4]   2> 3962617 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3962619 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3962622 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:52355",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3962659 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3962659 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3962661 INFO  (zkCallback-2143-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e95a39 name:ZooKeeperConnection Watcher:127.0.0.1:52518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3962661 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3962661 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3962661 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3962663 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3962724 INFO  (zkCallback-2143-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3962724 INFO  (zkCallback-2141-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3962772 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:52355/collection1/ and leader is https://127.0.0.1:52355/collection1/
   [junit4]   2> 3962772 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52355
   [junit4]   2> 3962772 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3962772 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3962772 INFO  (coreZkRegister-7036-thread-1-processing-n:127.0.0.1:52355_ c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:52355_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3962775 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:52355_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"https://127.0.0.1:52355",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 3962776 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:52355_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"https://127.0.0.1:52355",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 3962879 INFO  (zkCallback-2141-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3962879 INFO  (zkCallback-2143-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3962939 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/cores/collection1
   [junit4]   2> 3962940 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001
   [junit4]   2> 3962940 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3962942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a2512a3{/,null,AVAILABLE}
   [junit4]   2> 3962946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4ad1ddcc{SSL-http/1.1}{127.0.0.1:46707}
   [junit4]   2> 3962946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.e.j.s.Server Started @3966032ms
   [junit4]   2> 3962946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:58563/hdfs__localhost_58563__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001_tempDir-002_jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/cores, solrconfig=solrconfig.xml, hostContext=/, hostPort=46707}
   [junit4]   2> 3962946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3962946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/'
   [junit4]   2> 3962960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3962961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3962962 INFO  (zkCallback-2144-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7098202b name:ZooKeeperConnection Watcher:127.0.0.1:52518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3962962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3962963 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3962964 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3962964 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/solr.xml
   [junit4]   2> 3962977 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/cores
   [junit4]   2> 3962977 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1868331854
   [junit4]   2> 3962977 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/]
   [junit4]   2> 3962977 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/lib
   [junit4]   2> 3962977 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/lib).
   [junit4]   2> 3962985 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3962987 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3962989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3962989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3962989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52518/solr
   [junit4]   2> 3962989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3962989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3962990 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3962991 INFO  (zkCallback-2146-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@519f6bf7 name:ZooKeeperConnection Watcher:127.0.0.1:52518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3962991 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3962992 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3962993 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3962994 INFO  (zkCallback-2147-thread-1-processing-n:127.0.0.1:46707_) [n:127.0.0.1:46707_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1b54c825 name:ZooKeeperConnection Watcher:127.0.0.1:52518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3962994 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3962999 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3964005 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46707_
   [junit4]   2> 3964005 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46707_
   [junit4]   2> 3964007 INFO  (zkCallback-2143-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3964007 INFO  (zkCallback-2141-thread-1-processing-n:127.0.0.1:52355_) [n:127.0.0.1:52355_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3964007 INFO  (zkCallback-2147-thread-1-processing-n:127.0.0.1:46707_) [n:127.0.0.1:46707_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3964009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3964009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94567551427280905-127.0.0.1:46707_-n_0000000001
   [junit4]   2> 3964010 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94567551427280901-127.0.0.1:52355_-n_0000000000 to know if I could be the leader
   [junit4]   2> 3964027 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3964027 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3964028 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/cores
   [junit4]   2> 3964029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/cores/collection1/, transient=false, name=collection1, coreNodeName=, dataDir=data/, shard=, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/cores/collection1, configSetProperties=configsetprops.json}
   [junit4]   2> 3964029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_225CBFC8AC184BB8-001/shard-1-001/cores/collection1/
   [junit4]   2> 3964029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3964030 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3964030 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[225CBFC8AC184BB8]) [n:127.0.0.1:46707_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3964030 INFO  (coreLoadExecutor-7053-thread-1-processing-n:127.0.0.1:46707_) [n:127.0.0.1:46707_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3964030 INFO  (coreLoadExecutor-7053-thread-1-processing-n:127.0.0.1:46707_) [n:127.0.0.1:46707_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3964032 INFO  (coreLoadExecutor-7053-thread-1-processing-n:127.0.0.1:46707_) [n:127.0.0.1:46707_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3964032 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n_0000000000) [n:127.0.0.1:52355_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:46707_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"https://127.0.0.1:46707",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null} current state version: 3
   [junit4]   2> 3964032 INFO  (OverseerStateUpdate-94567551427280901-127.0.0.1:52355_-n

[...truncated too long message...]

ne-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.17.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150923_002555_558.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150923_002555_558.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1505)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 298 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1900658101 bytes
Compression is 0.0%
Took 5 min 48 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 963 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/963/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=78933, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=78933, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43264: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([DFADB585E81E4D08]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11403 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/init-core-data-001
   [junit4]   2> 4309811 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 4309812 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4309943 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4309964 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4309967 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4309991 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_58641_hdfs____.mcxvom/webapp
   [junit4]   2> 4310129 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4310670 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58641
   [junit4]   2> 4310931 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4310932 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4310945 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40518_datanode____jl87bb/webapp
   [junit4]   2> 4311107 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4311566 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40518
   [junit4]   2> 4311616 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4311618 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4311631 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53097_datanode____.rzrtih/webapp
   [junit4]   2> 4311809 INFO  (IPC Server handler 3 on 59103) [    ] BlockStateChange BLOCK* processReport: from storage DS-1776333d-8c49-4a45-b033-db04e7405532 node DatanodeRegistration(127.0.0.1, datanodeUuid=143b9a95-2872-499f-b2c1-6421ecc60f2a, infoPort=40518, ipcPort=45092, storageInfo=lv=-56;cid=testClusterID;nsid=1378124547;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 4311810 INFO  (IPC Server handler 3 on 59103) [    ] BlockStateChange BLOCK* processReport: from storage DS-5248d54a-4607-4c1b-9c44-4e8d6986647b node DatanodeRegistration(127.0.0.1, datanodeUuid=143b9a95-2872-499f-b2c1-6421ecc60f2a, infoPort=40518, ipcPort=45092, storageInfo=lv=-56;cid=testClusterID;nsid=1378124547;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4311837 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4312230 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DFADB585E81E4D08]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53097
   [junit4]   2> 4312414 INFO  (IPC Server handler 5 on 59103) [    ] BlockStateChange BLOCK* processReport: from storage DS-01d12f6c-5dce-46a2-8fb2-61d9bcda626f node DatanodeRegistration(127.0.0.1, datanodeUuid=f1f9e435-6ea7-484a-bc70-324da54ca000, infoPort=53097, ipcPort=59446, storageInfo=lv=-56;cid=testClusterID;nsid=1378124547;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 4312415 INFO  (IPC Server handler 5 on 59103) [    ] BlockStateChange BLOCK* processReport: from storage DS-651c764d-19e8-48c9-9773-e1bdfa6bccb9 node DatanodeRegistration(127.0.0.1, datanodeUuid=f1f9e435-6ea7-484a-bc70-324da54ca000, infoPort=53097, ipcPort=59446, storageInfo=lv=-56;cid=testClusterID;nsid=1378124547;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4312608 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4312610 INFO  (Thread-66182) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4312610 INFO  (Thread-66182) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4312710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.ZkTestServer start zk server on port:45713
   [junit4]   2> 4312710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4312729 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4312734 INFO  (zkCallback-2441-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@61d65b88 name:ZooKeeperConnection Watcher:127.0.0.1:45713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4312734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4312735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4312735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4312737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4312738 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4312740 INFO  (zkCallback-2442-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5725bebb name:ZooKeeperConnection Watcher:127.0.0.1:45713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4312740 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4312740 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4312740 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 4312742 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 4312744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 4312745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 4312747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4312747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4312750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4312750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 4312752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4312752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4312754 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4312754 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4312756 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4312756 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4312758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4312758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 4312760 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4312760 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4312761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4312762 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4312763 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4312764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4312765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4312766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4312767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4312768 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4313079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores/collection1
   [junit4]   2> 4313081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4313121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f4fb083{/,null,AVAILABLE}
   [junit4]   2> 4313122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3643e33e{HTTP/1.1}{127.0.0.1:43944}
   [junit4]   2> 4313122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.Server Started @4316649ms
   [junit4]   2> 4313122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores, hostPort=43944, hostContext=/, solr.data.dir=hdfs://localhost:59103/hdfs__localhost_59103__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001_tempDir-002_control_data}
   [junit4]   2> 4313123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 4313123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/'
   [junit4]   2> 4313148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4313169 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4313177 INFO  (zkCallback-2443-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@307c153d name:ZooKeeperConnection Watcher:127.0.0.1:45713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4313181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4313181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4313183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4313183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/solr.xml
   [junit4]   2> 4313248 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores
   [junit4]   2> 4313248 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoreContainer New CoreContainer 481483292
   [junit4]   2> 4313248 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/]
   [junit4]   2> 4313248 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/lib
   [junit4]   2> 4313249 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/lib).
   [junit4]   2> 4313259 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4313269 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4313270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4313270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4313270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45713/solr
   [junit4]   2> 4313270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4313270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4313279 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4313293 INFO  (zkCallback-2445-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20cf2cd name:ZooKeeperConnection Watcher:127.0.0.1:45713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4313293 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4313294 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4313314 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4313323 INFO  (zkCallback-2446-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b1de03 name:ZooKeeperConnection Watcher:127.0.0.1:45713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4313324 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4313325 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 4313326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 4313329 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 4313330 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 4313332 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 4313333 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 4313335 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 4313336 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 4313337 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 4313338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43944_
   [junit4]   2> 4313338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43944_
   [junit4]   2> 4313339 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 4313340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 4313341 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4313342 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94561244090269700-127.0.0.1:43944_-n_0000000000
   [junit4]   2> 4313342 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43944_
   [junit4]   2> 4313342 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 4313343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer Overseer (id=94561244090269700-127.0.0.1:43944_-n_0000000000) starting
   [junit4]   2> 4313345 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 4313350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4313358 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 4313358 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4313385 INFO  (OverseerCollectionConfigSetProcessor-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 4313392 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4313418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4313418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4313433 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores
   [junit4]   2> 4313434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=control_collection, config=solrconfig.xml, configSetProperties=configsetprops.json, schema=schema.xml, loadOnStartup=true, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores/collection1, dataDir=data/, shard=, name=collection1, transient=false}
   [junit4]   2> 4313434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores/collection1/
   [junit4]   2> 4313435 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4313436 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4313436 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4313438 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4313439 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:43944_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4313439 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4313439 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:43944",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:43944_",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 4313440 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:43944",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:43944_",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4313440 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4313440 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4313442 INFO  (zkCallback-2446-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4313442 INFO  (zkCallback-2446-thread-2-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4314440 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4314440 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 4314440 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4314441 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4314441 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4314441 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores/collection1/'
   [junit4]   2> 4314463 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4314475 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4314503 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 4314577 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4314582 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 4314592 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4315005 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4315023 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 4315025 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 4315046 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4315051 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4315055 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4315057 WARN  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4315058 WARN  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4315058 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4315059 WARN  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4315059 WARN  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4315060 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4315060 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4315061 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 4315061 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:59103/solr_hdfs_home
   [junit4]   2> 4315061 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4315061 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 4315061 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@f64cb82
   [junit4]   2> 4315062 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4315097 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 4315097 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4315099 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 4315127 WARN  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:59103/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4315127 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4315161 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 4315161 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4315162 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=89.1748046875, floorSegmentMB=1.1328125, forceMergeDeletesPctAllowed=19.94707717890828, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 4315247 INFO  (IPC Server handler 3 on 59103) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35229 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-01d12f6c-5dce-46a2-8fb2-61d9bcda626f:NORMAL:127.0.0.1:41564|RBW], ReplicaUnderConstruction[[DISK]DS-5248d54a-4607-4c1b-9c44-4e8d6986647b:NORMAL:127.0.0.1:35229|FINALIZED]]} size 0
   [junit4]   2> 4315250 INFO  (IPC Server handler 4 on 59103) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41564 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5248d54a-4607-4c1b-9c44-4e8d6986647b:NORMAL:127.0.0.1:35229|FINALIZED], ReplicaUnderConstruction[[DISK]DS-651c764d-19e8-48c9-9773-e1bdfa6bccb9:NORMAL:127.0.0.1:41564|FINALIZED]]} size 0
   [junit4]   2> 4315253 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b9070039 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@15b64d8e; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4315253 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4315265 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:59103/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 4315269 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4315269 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4315269 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4315269 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4315269 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4315269 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4315270 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4315270 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4315270 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4315273 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4315274 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4315276 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4315289 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4315293 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/ping,/update/json,/update,/admin/file,/replication,/admin/threads,/admin/logging,/admin/segments,/admin/mbeans,/admin/luke,/update/csv,/config,/admin/plugins,/update/json/docs,/admin/properties,/schema,/get,standard,/admin/system
   [junit4]   2> 4315295 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 4315305 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4315305 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 4315326 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 4315326 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 4315327 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=30, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8537218780375183]
   [junit4]   2> 4315336 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b9070039 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@15b64d8e; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4315336 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4315336 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@22337f30[collection1] main
   [junit4]   2> 4315336 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4315337 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4315337 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4315338 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4315338 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 4315338 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4315338 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4315339 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4315339 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 4315339 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4315340 INFO  (searcherExecutor-8291-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@22337f30[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4315342 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 4315342 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 4315342 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 4315342 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1512979908301684736
   [junit4]   2> 4315342 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1512979908301684736
   [junit4]   2> 4315342 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 4315343 INFO  (coreLoadExecutor-8290-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 4315344 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:43944 collection:control_collection shard:shard1
   [junit4]   2> 4315344 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 4315348 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94561244090269700-core_node1-n_0000000000
   [junit4]   2> 4315348 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 4315350 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4315350 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 4315350 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4315351 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43944/collection1/
   [junit4]   2> 4315351 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4315352 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43944/collection1/ has no replicas
   [junit4]   2> 4315351 INFO  (zkCallback-2446-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4315351 INFO  (zkCallback-2446-thread-2-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4315352 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43944/collection1/ shard1
   [junit4]   2> 4315354 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 4315356 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 4315359 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43944",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 4315441 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4315442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4315443 INFO  (zkCallback-2448-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d64e5bd name:ZooKeeperConnection Watcher:127.0.0.1:45713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4315444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4315444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4315444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4315461 INFO  (zkCallback-2446-thread-2-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4315461 INFO  (zkCallback-2446-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4315464 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 4315509 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:43944/collection1/ and leader is http://127.0.0.1:43944/collection1/
   [junit4]   2> 4315509 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43944
   [junit4]   2> 4315509 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4315510 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 4315510 INFO  (coreZkRegister-8284-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:43944_) [n:127.0.0.1:43944_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4315512 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:43944",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:43944_",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 4315513 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:43944",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:43944_",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4315615 INFO  (zkCallback-2446-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4315615 INFO  (zkCallback-2446-thread-2-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4315615 INFO  (zkCallback-2448-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4315836 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores/collection1
   [junit4]   2> 4315837 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001
   [junit4]   2> 4315838 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4315901 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ff2e619{/,null,AVAILABLE}
   [junit4]   2> 4315902 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.ServerConnector Started ServerConnector@33e0acba{HTTP/1.1}{127.0.0.1:47928}
   [junit4]   2> 4315902 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.e.j.s.Server Started @4319429ms
   [junit4]   2> 4315902 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=47928, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:59103/hdfs__localhost_59103__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/}
   [junit4]   2> 4315902 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 4315903 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/'
   [junit4]   2> 4315926 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4315941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4315969 INFO  (zkCallback-2449-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b283fa9 name:ZooKeeperConnection Watcher:127.0.0.1:45713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4315969 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4315970 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4315971 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4315971 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/solr.xml
   [junit4]   2> 4315985 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores
   [junit4]   2> 4315985 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoreContainer New CoreContainer 1915646554
   [junit4]   2> 4315986 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/]
   [junit4]   2> 4315986 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/lib
   [junit4]   2> 4315986 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/lib).
   [junit4]   2> 4315993 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4315995 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4315995 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4315995 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4315996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45713/solr
   [junit4]   2> 4315996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4315996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4315997 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4315998 INFO  (zkCallback-2451-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a5bead7 name:ZooKeeperConnection Watcher:127.0.0.1:45713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4315998 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4315998 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4316000 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4316001 INFO  (zkCallback-2452-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38515b6f name:ZooKeeperConnection Watcher:127.0.0.1:45713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4316001 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4316006 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4317011 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47928_
   [junit4]   2> 4317012 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47928_
   [junit4]   2> 4317013 INFO  (zkCallback-2448-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4317013 INFO  (zkCallback-2446-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4317013 INFO  (zkCallback-2452-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4317013 INFO  (zkCallback-2446-thread-2-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4317014 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4317016 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94561244090269704-127.0.0.1:47928_-n_0000000001
   [junit4]   2> 4317016 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94561244090269700-127.0.0.1:43944_-n_0000000000 to know if I could be the leader
   [junit4]   2> 4317042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4317042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4317053 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores
   [junit4]   2> 4317054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores/collection1, transient=false, loadOnStartup=true, coreNodeName=, config=solrconfig.xml, name=collection1, dataDir=data/, configSetProperties=configsetprops.json, schema=schema.xml, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores/collection1/}
   [junit4]   2> 4317054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores/collection1/
   [junit4]   2> 4317054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4317074 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4317074 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4317076 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47928",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:47928_",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 4317076 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4317076 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47928",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:47928_",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4317077 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 4317077 INFO  (OverseerStateUpdate-94561244090269700-127.0.0.1:43944_-n_0000000000) [n:127.0.0.1:43944_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 4317081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4317081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DFADB585E81E4D08]) [n:127.0.0.1:47928_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4317180 INFO  (zkCallback-2452-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4317180 INFO  (zkCallback-2446-thread-1-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4317180 INFO  (zkCallback-2446-thread-2-processing-n:127.0.0.1:43944_) [n:127.0.0.1:43944_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4317180 INFO  (zkCallback-2448-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4318077 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4318078 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 4318078 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4318079 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 4318079 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4318079 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DFADB585E81E4D08-001/shard-1-001/cores/collection1/'
   [junit4]   2> 4318118 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4318126 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4318146 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 4318206 INFO  (coreLoadExecutor-8301-thread-1-processing-n:127.0.0.1:47928_) [n:127.0.0.1:47928_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: 

[...truncated too long message...]

sionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:499)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 7352254 INFO  (qtp161958140-11593) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection0_1 s:shard2 r:core_node5 x:awholynewstresscollection_collection0_1_shard2_replica1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 19483020 INFO  (qtp161958140-11593) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection0_1 s:shard2 r:core_node5 x:awholynewstresscollection_collection0_1_shard2_replica1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 19483020 INFO  (qtp161958140-11593) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection0_1 s:shard2 r:core_node5 x:awholynewstresscollection_collection0_1_shard2_replica1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 19483021 INFO  (qtp161958140-11593) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection0_1 s:shard2 r:core_node5 x:awholynewstresscollection_collection0_1_shard2_replica1] o.a.s.c.SolrCore [awholynewstresscollection_collection0_1_shard2_replica1] Closing main searcher on request.
   [junit4]   2> 7237449 INFO  (qtp645767994-11381) [n:127.0.0.1:39676_ c:awholynewstresscollection_collection1_1 s:shard5 r:core_node17 x:awholynewstresscollection_collection1_1_shard5_replica1] o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores params={collection.configName=conf1&numShards=5&collection=awholynewstresscollection_collection1_1&action=CREATE&version=2&shard=shard5&name=awholynewstresscollection_collection1_1_shard5_replica1&wt=javabin&qt=/admin/cores} status=0 QTime=4658330 
   [junit4]   2> 7233641 INFO  (zkCallback-1752-thread-18-processing-n:127.0.0.1:46109_) [n:127.0.0.1:46109_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 7208845 INFO  (qtp161958140-11585) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection2_1 s:shard5 r:core_node8 x:awholynewstresscollection_collection2_1_shard5_replica1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 19485823 INFO  (qtp161958140-11585) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection2_1 s:shard5 r:core_node8 x:awholynewstresscollection_collection2_1_shard5_replica1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 19486379 INFO  (qtp161958140-11585) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection2_1 s:shard5 r:core_node8 x:awholynewstresscollection_collection2_1_shard5_replica1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 19486380 INFO  (qtp161958140-11585) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection2_1 s:shard5 r:core_node8 x:awholynewstresscollection_collection2_1_shard5_replica1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 19487321 INFO  (qtp161958140-11585) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection2_1 s:shard5 r:core_node8 x:awholynewstresscollection_collection2_1_shard5_replica1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 19487321 INFO  (qtp161958140-11585) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection2_1 s:shard5 r:core_node8 x:awholynewstresscollection_collection2_1_shard5_replica1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 19487321 INFO  (qtp161958140-11585) [n:127.0.0.1:46109_ c:awholynewstresscollection_collection2_1 s:shard5 r:core_node8 x:awholynewstresscollection_collection2_1_shard5_replica1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 6924527 INFO  (qtp645767994-11295) [n:127.0.0.1:39676_ c:awholynewstresscollection_collection0_1 s:shard1 r:core_node11 x:awholynewstresscollection_collection0_1_shard1_replica4] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=nl_NL, timezone=Africa/Tripoli
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=269,free=42991456,total=471859200
   [junit4]   2> NOTE: All tests run in this JVM: [TestBinaryResponseWriter, HttpPartitionTest, ConnectionManagerTest, TestRuleBasedAuthorizationPlugin, TestEmbeddedSolrServerConstructors, DistanceUnitsTest, TestHashQParserPlugin, TestRTGBase, TestSolrCoreProperties, SSLMigrationTest, ZkControllerTest, TestLMJelinekMercerSimilarityFactory, SaslZkACLProviderTest, SchemaVersionSpecificBehaviorTest, ZkCLITest, SortByFunctionTest, HdfsThreadLeakTest, TestCustomSort, ZkStateReaderTest, TestImplicitCoreProperties, IgnoreCommitOptimizeUpdateProcessorFactoryTest, OverseerRolesTest, TestTolerantSearch, TestJmxMonitoredMap, TestCustomDocTransformer, SpatialFilterTest, SpellCheckComponentTest, CachingDirectoryFactoryTest, TestBM25SimilarityFactory, TestConfigSets, AnalyticsQueryTest, OverriddenZkACLAndCredentialsProvidersTest, DocValuesMissingTest, TestFieldTypeCollectionResource, TestUtils, TestSimpleQParserPlugin, ConvertedLegacyTest, DirectSolrSpellCheckerTest, AddSchemaFieldsUpdateProcessorFactoryTest, RuleEngineTest, TestFastOutputStream, SolrPluginUtilsTest, TestMiniSolrCloudCluster, PreAnalyzedFieldTest, SolrIndexSplitterTest, TestReloadDeadlock, TestHighFrequencyDictionaryFactory, SolrCmdDistributorTest, HighlighterTest, CoreAdminRequestStatusTest, TestWordDelimiterFilterFactory, SpellPossibilityIteratorTest, TestMissingGroups, TestLFUCache, TestFiltering, SolrCloudExampleTest, TestPivotHelperCode, TestCloudManagedSchemaConcurrent, OutOfBoxZkACLAndCredentialsProvidersTest, SoftAutoCommitTest, BasicFunctionalityTest, SolrInfoMBeanTest, OpenCloseCoreStressTest, TestPKIAuthenticationPlugin, WordBreakSolrSpellCheckerTest, TestIntervalFaceting, HdfsSyncSliceTest, ChaosMonkeyNothingIsSafeTest, TestUniqueKeyFieldResource, TestSolrConfigHandler, TestCloudSchemaless, TestConfigSetImmutable, TestUpdate, SimplePostToolTest, TestSearcherReuse, DeleteLastCustomShardedReplicaTest, SegmentsInfoRequestHandlerTest, MoreLikeThisHandlerTest, MBeansHandlerTest, CircularListTest, MinimalSchemaTest, TestBlobHandler, TestSchemaResource, OverseerTaskQueueTest, ExitableDirectoryReaderTest, TestCodecSupport, TestCloudManagedSchema, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] ERROR   0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4] Completed [539/539] on J1 in 17022.57s, 1 test, 1 error <<< FAILURES!

[...truncated 2 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150921_213600_802.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid21935.hprof ...
   [junit4] Heap dump file created [631638425 bytes in 17.637 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 11 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites, 2128 tests, 1 suite-level error, 1 error, 29 ignored (17 assumptions)

Total time: 445 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1874068156 bytes
Compression is 0.0%
Took 5 min 40 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 962 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/962/

6 tests failed.
REGRESSION:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#6): CloudJettyRunner [url=http://127.0.0.1:57840/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#6): CloudJettyRunner [url=http://127.0.0.1:57840/collection1]
	at __randomizedtesting.SeedInfo.seed([8B4E93F72888A614:31AAC2D8674CBEC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:36499/or/o/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:36499/or/o/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([8B4E93F72888A614:31AAC2D8674CBEC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.core.OpenCloseCoreStressTest.test10MinutesOld

Error Message:
Captured an uncaught exception in thread: Thread[id=3900, name=Lucene Merge Thread #67, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3900, name=Lucene Merge Thread #67, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-028/_22r_Asserting_0.tip: Too many open files
	at __randomizedtesting.SeedInfo.seed([8B4E93F72888A614]:0)
	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-028/_22r_Asserting_0.tip: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:198)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:334)
	at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:82)
	at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:190)
	at org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2541)
	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:640)
	at org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:173)
	at org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:446)
	at org.apache.lucene.codecs.asserting.AssertingPostingsFormat.fieldsProducer(AssertingPostingsFormat.java:59)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3827)
	at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3785)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3639)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1442804497301,generation=2,filelist=[_10o.cfe, _10o.cfs, _10o.si, _11s.cfe, _11s.cfs, _11s.si, _12w.cfe, _12w.cfs, _12w.si, _14.cfe, _14.cfs, _14.si, _140.cfe, _140.cfs, _140.si, _154.cfe, _154.cfs, _154.si, _168.cfe, _168.cfs, _168.si, _17b.cfe, _17b.cfs, _17b.si, _17c.cfe, _17c.cfs, _17c.si, _17e.cfe, _17e.cfs, _17e.si, _17f.cfe, _17f.cfs, _17f.si, _17g.cfe, _17g.cfs, _17g.si, _17h.cfe, _17h.cfs, _17h.si, _17i.cfe, _17i.cfs, _17i.si, _28.cfe, _28.cfs, _28.si, _3c.cfe, _3c.cfs, _3c.si, _4g.cfe, _4g.cfs, _4g.si, _5k.cfe, _5k.cfs, _5k.si, _6o.cfe, _6o.cfs, _6o.si, _7s.cfe, _7s.cfs, _7s.si, _8w.cfe, _8w.cfs, _8w.si, _a0.cfe, _a0.cfs, _a0.si, _b4.cfe, _b4.cfs, _b4.si, _c8.cfe, _c8.cfs, _c8.si, _dc.cfe, _dc.cfs, _dc.si, _eg.cfe, _eg.cfs, _eg.si, _fk.cfe, _fk.cfs, _fk.si, _go.cfe, _go.cfs, _go.si, _hs.cfe, _hs.cfs, _hs.si, _iw.cfe, _iw.cfs, _iw.si, _k0.cfe, _k0.cfs, _k0.si, _l4.cfe, _l4.cfs, _l4.si, _m8.cfe, _m8.cfs, _m8.si, _nc.cfe, _nc.cfs, _nc.si, _og.cfe, _og.cfs, _og.si, _pk.cfe, _pk.cfs, _pk.si, _qo.cfe, _qo.cfs, _qo.si, _rs.cfe, _rs.cfs, _rs.si, _sw.cfe, _sw.cfs, _sw.si, _u0.cfe, _u0.cfs, _u0.si, _v4.cfe, _v4.cfs, _v4.si, _w8.cfe, _w8.cfs, _w8.si, _xc.cfe, _xc.cfs, _xc.si, _yg.cfe, _yg.cfs, _yg.si, _zk.cfe, _zk.cfs, _zk.si, segments_2]}]> but was:<[{indexVersion=1442804497301,generation=2,filelist=[_10o.cfe, _10o.cfs, _10o.si, _11s.cfe, _11s.cfs, _11s.si, _12w.cfe, _12w.cfs, _12w.si, _14.cfe, _14.cfs, _14.si, _140.cfe, _140.cfs, _140.si, _154.cfe, _154.cfs, _154.si, _168.cfe, _168.cfs, _168.si, _17b.cfe, _17b.cfs, _17b.si, _17c.cfe, _17c.cfs, _17c.si, _17e.cfe, _17e.cfs, _17e.si, _17f.cfe, _17f.cfs, _17f.si, _17g.cfe, _17g.cfs, _17g.si, _17h.cfe, _17h.cfs, _17h.si, _17i.cfe, _17i.cfs, _17i.si, _28.cfe, _28.cfs, _28.si, _3c.cfe, _3c.cfs, _3c.si, _4g.cfe, _4g.cfs, _4g.si, _5k.cfe, _5k.cfs, _5k.si, _6o.cfe, _6o.cfs, _6o.si, _7s.cfe, _7s.cfs, _7s.si, _8w.cfe, _8w.cfs, _8w.si, _a0.cfe, _a0.cfs, _a0.si, _b4.cfe, _b4.cfs, _b4.si, _c8.cfe, _c8.cfs, _c8.si, _dc.cfe, _dc.cfs, _dc.si, _eg.cfe, _eg.cfs, _eg.si, _fk.cfe, _fk.cfs, _fk.si, _go.cfe, _go.cfs, _go.si, _hs.cfe, _hs.cfs, _hs.si, _iw.cfe, _iw.cfs, _iw.si, _k0.cfe, _k0.cfs, _k0.si, _l4.cfe, _l4.cfs, _l4.si, _m8.cfe, _m8.cfs, _m8.si, _nc.cfe, _nc.cfs, _nc.si, _og.cfe, _og.cfs, _og.si, _pk.cfe, _pk.cfs, _pk.si, _qo.cfe, _qo.cfs, _qo.si, _rs.cfe, _rs.cfs, _rs.si, _sw.cfe, _sw.cfs, _sw.si, _u0.cfe, _u0.cfs, _u0.si, _v4.cfe, _v4.cfs, _v4.si, _w8.cfe, _w8.cfs, _w8.si, _xc.cfe, _xc.cfs, _xc.si, _yg.cfe, _yg.cfs, _yg.si, _zk.cfe, _zk.cfs, _zk.si, segments_2]}, {indexVersion=1442804497301,generation=3,filelist=[_17b.cfe, _17b.cfs, _17b.si, _17d.cfe, _17d.cfs, _17d.si, _17e.cfe, _17e.cfs, _17e.si, _17f.cfe, _17f.cfs, _17f.si, _17g.cfe, _17g.cfs, _17g.si, _17h.cfe, _17h.cfs, _17h.si, _17i.cfe, _17i.cfs, _17i.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1442804497301,generation=2,filelist=[_10o.cfe, _10o.cfs, _10o.si, _11s.cfe, _11s.cfs, _11s.si, _12w.cfe, _12w.cfs, _12w.si, _14.cfe, _14.cfs, _14.si, _140.cfe, _140.cfs, _140.si, _154.cfe, _154.cfs, _154.si, _168.cfe, _168.cfs, _168.si, _17b.cfe, _17b.cfs, _17b.si, _17c.cfe, _17c.cfs, _17c.si, _17e.cfe, _17e.cfs, _17e.si, _17f.cfe, _17f.cfs, _17f.si, _17g.cfe, _17g.cfs, _17g.si, _17h.cfe, _17h.cfs, _17h.si, _17i.cfe, _17i.cfs, _17i.si, _28.cfe, _28.cfs, _28.si, _3c.cfe, _3c.cfs, _3c.si, _4g.cfe, _4g.cfs, _4g.si, _5k.cfe, _5k.cfs, _5k.si, _6o.cfe, _6o.cfs, _6o.si, _7s.cfe, _7s.cfs, _7s.si, _8w.cfe, _8w.cfs, _8w.si, _a0.cfe, _a0.cfs, _a0.si, _b4.cfe, _b4.cfs, _b4.si, _c8.cfe, _c8.cfs, _c8.si, _dc.cfe, _dc.cfs, _dc.si, _eg.cfe, _eg.cfs, _eg.si, _fk.cfe, _fk.cfs, _fk.si, _go.cfe, _go.cfs, _go.si, _hs.cfe, _hs.cfs, _hs.si, _iw.cfe, _iw.cfs, _iw.si, _k0.cfe, _k0.cfs, _k0.si, _l4.cfe, _l4.cfs, _l4.si, _m8.cfe, _m8.cfs, _m8.si, _nc.cfe, _nc.cfs, _nc.si, _og.cfe, _og.cfs, _og.si, _pk.cfe, _pk.cfs, _pk.si, _qo.cfe, _qo.cfs, _qo.si, _rs.cfe, _rs.cfs, _rs.si, _sw.cfe, _sw.cfs, _sw.si, _u0.cfe, _u0.cfs, _u0.si, _v4.cfe, _v4.cfs, _v4.si, _w8.cfe, _w8.cfs, _w8.si, _xc.cfe, _xc.cfs, _xc.si, _yg.cfe, _yg.cfs, _yg.si, _zk.cfe, _zk.cfs, _zk.si, segments_2]}]> but was:<[{indexVersion=1442804497301,generation=2,filelist=[_10o.cfe, _10o.cfs, _10o.si, _11s.cfe, _11s.cfs, _11s.si, _12w.cfe, _12w.cfs, _12w.si, _14.cfe, _14.cfs, _14.si, _140.cfe, _140.cfs, _140.si, _154.cfe, _154.cfs, _154.si, _168.cfe, _168.cfs, _168.si, _17b.cfe, _17b.cfs, _17b.si, _17c.cfe, _17c.cfs, _17c.si, _17e.cfe, _17e.cfs, _17e.si, _17f.cfe, _17f.cfs, _17f.si, _17g.cfe, _17g.cfs, _17g.si, _17h.cfe, _17h.cfs, _17h.si, _17i.cfe, _17i.cfs, _17i.si, _28.cfe, _28.cfs, _28.si, _3c.cfe, _3c.cfs, _3c.si, _4g.cfe, _4g.cfs, _4g.si, _5k.cfe, _5k.cfs, _5k.si, _6o.cfe, _6o.cfs, _6o.si, _7s.cfe, _7s.cfs, _7s.si, _8w.cfe, _8w.cfs, _8w.si, _a0.cfe, _a0.cfs, _a0.si, _b4.cfe, _b4.cfs, _b4.si, _c8.cfe, _c8.cfs, _c8.si, _dc.cfe, _dc.cfs, _dc.si, _eg.cfe, _eg.cfs, _eg.si, _fk.cfe, _fk.cfs, _fk.si, _go.cfe, _go.cfs, _go.si, _hs.cfe, _hs.cfs, _hs.si, _iw.cfe, _iw.cfs, _iw.si, _k0.cfe, _k0.cfs, _k0.si, _l4.cfe, _l4.cfs, _l4.si, _m8.cfe, _m8.cfs, _m8.si, _nc.cfe, _nc.cfs, _nc.si, _og.cfe, _og.cfs, _og.si, _pk.cfe, _pk.cfs, _pk.si, _qo.cfe, _qo.cfs, _qo.si, _rs.cfe, _rs.cfs, _rs.si, _sw.cfe, _sw.cfs, _sw.si, _u0.cfe, _u0.cfs, _u0.si, _v4.cfe, _v4.cfs, _v4.si, _w8.cfe, _w8.cfs, _w8.si, _xc.cfe, _xc.cfs, _xc.si, _yg.cfe, _yg.cfs, _yg.si, _zk.cfe, _zk.cfs, _zk.si, segments_2]}, {indexVersion=1442804497301,generation=3,filelist=[_17b.cfe, _17b.cfs, _17b.si, _17d.cfe, _17d.cfs, _17d.si, _17e.cfe, _17e.cfs, _17e.si, _17f.cfe, _17f.cfs, _17f.si, _17g.cfe, _17g.cfs, _17g.si, _17h.cfe, _17h.cfs, _17h.si, _17i.cfe, _17i.cfs, _17i.si, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([8B4E93F72888A614:AE9988C758C0A817]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=8468, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8468, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40886/wwo/ha: collection already exists: awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([8B4E93F72888A614]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=18839, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=18839, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33503/_t: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([8B4E93F72888A614]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10233 lines...]
   [junit4] Suite: org.apache.solr.core.OpenCloseCoreStressTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/init-core-data-001
   [junit4]   2> 40871 INFO  (SUITE-OpenCloseCoreStressTest-seed#[8B4E93F72888A614]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 40877 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test5Seconds
   [junit4]   2> 40891 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 40893 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@8fe70f7{/solr,null,AVAILABLE}
   [junit4]   2> 40896 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.e.j.s.ServerConnector Started ServerConnector@78aa6cbd{HTTP/1.1}{127.0.0.1:40361}
   [junit4]   2> 40897 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.e.j.s.Server Started @43882ms
   [junit4]   2> 40897 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=40361, hostContext=/solr}
   [junit4]   2> 40897 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 40898 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/'
   [junit4]   2> 40915 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/solr.xml
   [junit4]   2> 40930 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001
   [junit4]   2> 40930 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoreContainer New CoreContainer 653470580
   [junit4]   2> 40931 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/]
   [junit4]   2> 40931 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/lib
   [junit4]   2> 40931 WARN  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/lib).
   [junit4]   2> 40940 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 40941 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 40942 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 40942 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 40943 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 40943 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 40944 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001
   [junit4]   2> 40945 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=false, configSetProperties=configsetprops.json, transient=true, name=00000_core, schema=schema-tiny.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/, config=solrconfig-minimal.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core, dataDir=data/}
   [junit4]   2> 40945 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found core 00000_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/
   [junit4]   2> 40946 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, name=00004_core, transient=true, schema=schema-tiny.xml, loadOnStartup=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core, config=solrconfig-minimal.xml, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/}
   [junit4]   2> 40946 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found core 00004_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/
   [junit4]   2> 40947 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema-tiny.xml, transient=true, name=00003_core, configSetProperties=configsetprops.json, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/, config=solrconfig-minimal.xml, loadOnStartup=false}
   [junit4]   2> 40947 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found core 00003_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/
   [junit4]   2> 40948 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00002_core, schema=schema-tiny.xml, name=00002_core, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00002_core/, configSetProperties=configsetprops.json, loadOnStartup=false, dataDir=data/, transient=true, config=solrconfig-minimal.xml}
   [junit4]   2> 40948 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found core 00002_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00002_core/
   [junit4]   2> 40948 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/, dataDir=data/, configSetProperties=configsetprops.json, name=00006_core, config=solrconfig-minimal.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core, loadOnStartup=false, schema=schema-tiny.xml}
   [junit4]   2> 40949 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found core 00006_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/
   [junit4]   2> 40949 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00001_core, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00001_core/, config=solrconfig-minimal.xml, name=00001_core, loadOnStartup=false, schema=schema-tiny.xml, dataDir=data/, configSetProperties=configsetprops.json, transient=true}
   [junit4]   2> 40949 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found core 00001_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00001_core/
   [junit4]   2> 40950 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema-tiny.xml, configSetProperties=configsetprops.json, loadOnStartup=false, config=solrconfig-minimal.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/, name=00005_core, transient=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core, dataDir=data/}
   [junit4]   2> 40950 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found core 00005_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/
   [junit4]   2> 40950 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.c.CoresLocator Found 7 core definitions
   [junit4]   2> 40951 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 40951 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 40962 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.SolrTestCaseJ4 Starting indexing and querying
   [junit4]   2> 40963 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[8B4E93F72888A614]) [    ] o.a.s.SolrTestCaseJ4 
   [junit4]   2> 
   [junit4]   2> 
   [junit4]   2> 
   [junit4]   2> 
   [junit4]   2> Starting a 5 second cycle, seconds left: 5. Seconds run so far: 0.
   [junit4]   2> 40964 INFO  (Thread-85) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 380
   [junit4]   2> 40964 INFO  (Thread-86) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 381
   [junit4]   2> 40965 INFO  (Thread-88) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 383
   [junit4]   2> 40966 INFO  (Thread-91) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 386
   [junit4]   2> 40965 INFO  (Thread-87) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 382
   [junit4]   2> 40965 INFO  (Thread-89) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 384
   [junit4]   2> 40967 INFO  (Thread-92) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 387
   [junit4]   2> 40965 INFO  (Thread-90) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 385
   [junit4]   2> 40970 INFO  (Thread-93) [    ] o.a.s.SolrTestCaseJ4 Starting indexing thread: 388
   [junit4]   2> 40972 INFO  (Thread-94) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 389
   [junit4]   2> 40972 INFO  (Thread-95) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 390
   [junit4]   2> 40972 INFO  (Thread-96) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 391
   [junit4]   2> 40973 INFO  (Thread-97) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 392
   [junit4]   2> 40973 INFO  (Thread-98) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 393
   [junit4]   2> 40973 INFO  (Thread-100) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 395
   [junit4]   2> 40976 INFO  (Thread-99) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 394
   [junit4]   2> 40976 INFO  (Thread-101) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 396
   [junit4]   2> 40977 INFO  (Thread-102) [    ] o.a.s.SolrTestCaseJ4 Starting query thread: 397
   [junit4]   2> 40983 INFO  (qtp1120754047-370) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/'
   [junit4]   2> 40983 INFO  (qtp1120754047-372) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/'
   [junit4]   2> 40983 INFO  (qtp1120754047-371) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/'
   [junit4]   2> 40985 INFO  (qtp1120754047-399) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/'
   [junit4]   2> 41022 INFO  (qtp1120754047-399) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 41039 INFO  (qtp1120754047-401) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/'
   [junit4]   2> 41042 INFO  (qtp1120754047-399) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 41044 INFO  (qtp1120754047-370) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 41052 INFO  (qtp1120754047-371) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 41059 INFO  (qtp1120754047-372) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 41060 INFO  (qtp1120754047-370) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 41109 INFO  (qtp1120754047-372) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 41123 INFO  (qtp1120754047-401) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 41123 INFO  (qtp1120754047-371) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 41146 INFO  (qtp1120754047-399) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 41147 INFO  (qtp1120754047-399) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/conf/schema-tiny.xml
   [junit4]   2> 41148 INFO  (qtp1120754047-401) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 41171 INFO  (qtp1120754047-370) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 41171 INFO  (qtp1120754047-370) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/conf/schema-tiny.xml
   [junit4]   2> 41177 INFO  (qtp1120754047-370) [    ] o.a.s.s.IndexSchema [00005_core] Schema name=tiny
   [junit4]   2> 41180 INFO  (qtp1120754047-399) [    ] o.a.s.s.IndexSchema [00004_core] Schema name=tiny
   [junit4]   2> 41191 INFO  (qtp1120754047-372) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 41192 INFO  (qtp1120754047-372) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/conf/schema-tiny.xml
   [junit4]   2> 41219 INFO  (qtp1120754047-407) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00002_core/'
   [junit4]   2> 41227 INFO  (qtp1120754047-399) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 41229 INFO  (qtp1120754047-399) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/conf'
   [junit4]   2> 41229 INFO  (qtp1120754047-399) [    ] o.a.s.c.CoreContainer Creating SolrCore '00004_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/
   [junit4]   2> 41229 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 41230 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore [[00004_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/], dataDir=[null]
   [junit4]   2> 41230 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore JMX monitoring not detected for core: 00004_core
   [junit4]   2> 41231 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/data
   [junit4]   2> 41234 INFO  (qtp1120754047-371) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 41235 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/data/index/
   [junit4]   2> 41235 WARN  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore [00004_core] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 41239 INFO  (qtp1120754047-371) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/conf/schema-tiny.xml
   [junit4]   2> 41246 INFO  (qtp1120754047-370) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 41267 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/data/index
   [junit4]   2> 41268 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 41271 INFO  (qtp1120754047-371) [    ] o.a.s.s.IndexSchema [00003_core] Schema name=tiny
   [junit4]   2> 41272 INFO  (qtp1120754047-407) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 41275 INFO  (qtp1120754047-372) [    ] o.a.s.s.IndexSchema [00000_core] Schema name=tiny
   [junit4]   2> 41279 INFO  (qtp1120754047-370) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/conf'
   [junit4]   2> 41279 INFO  (qtp1120754047-370) [    ] o.a.s.c.CoreContainer Creating SolrCore '00005_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/
   [junit4]   2> 41279 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 41280 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore [[00005_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/], dataDir=[null]
   [junit4]   2> 41280 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore JMX monitoring not detected for core: 00005_core
   [junit4]   2> 41292 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/data
   [junit4]   2> 41293 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/data/index/
   [junit4]   2> 41293 WARN  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore [00005_core] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 41311 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/data/index
   [junit4]   2> 41311 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 41304 INFO  (qtp1120754047-401) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 41313 INFO  (qtp1120754047-401) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/conf/schema-tiny.xml
   [junit4]   2> 41302 INFO  (qtp1120754047-372) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 41315 INFO  (qtp1120754047-372) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/conf'
   [junit4]   2> 41315 INFO  (qtp1120754047-372) [    ] o.a.s.c.CoreContainer Creating SolrCore '00000_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/
   [junit4]   2> 41315 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 41315 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore [[00000_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/], dataDir=[null]
   [junit4]   2> 41316 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore JMX monitoring not detected for core: 00000_core
   [junit4]   2> 41317 INFO  (qtp1120754047-407) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 41318 INFO  (qtp1120754047-401) [    ] o.a.s.s.IndexSchema [00006_core] Schema name=tiny
   [junit4]   2> 41320 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/data
   [junit4]   2> 41320 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/data/index/
   [junit4]   2> 41321 WARN  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore [00000_core] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 41321 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/data/index
   [junit4]   2> 41322 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 41323 INFO  (qtp1120754047-371) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 41328 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41328 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41331 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41331 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41331 INFO  (qtp1120754047-371) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/conf'
   [junit4]   2> 41334 INFO  (qtp1120754047-371) [    ] o.a.s.c.CoreContainer Creating SolrCore '00003_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/
   [junit4]   2> 41334 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 41334 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore [[00003_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/], dataDir=[null]
   [junit4]   2> 41334 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore JMX monitoring not detected for core: 00003_core
   [junit4]   2> 41341 INFO  (qtp1120754047-401) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 41342 INFO  (qtp1120754047-401) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/conf'
   [junit4]   2> 41343 INFO  (qtp1120754047-401) [    ] o.a.s.c.CoreContainer Creating SolrCore '00006_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/
   [junit4]   2> 41346 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-003 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41346 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41343 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/data
   [junit4]   2> 41347 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/data/index/
   [junit4]   2> 41347 WARN  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore [00003_core] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 41347 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 41348 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore [[00006_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/], dataDir=[null]
   [junit4]   2> 41348 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore JMX monitoring not detected for core: 00006_core
   [junit4]   2> 41348 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/data/index
   [junit4]   2> 41348 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/data
   [junit4]   2> 41348 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 41349 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/data/index/
   [junit4]   2> 41349 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 41349 WARN  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore [00006_core] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 41351 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/data/index
   [junit4]   2> 41351 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 41352 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41353 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41353 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41353 INFO  (OldIndexDirectoryCleanupThreadForCore-00005_core) [    x:00005_core] o.a.s.c.SolrCore Looking for old index directories to cleanup for core 00005_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/data/
   [junit4]   2> 41354 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41363 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 41354 WARN  (OldIndexDirectoryCleanupThreadForCore-00005_core) [    x:00005_core] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 41366 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41367 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41369 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.RequestHandlers Registered paths: /select,/replication,/admin/threads,/admin/file,/update/csv,/admin/plugins,/get,/admin/properties,/admin/logging,/update/json,/admin/ping,/admin/system,/update,/admin/luke,/config,/admin/segments,/admin/mbeans,/schema,/update/json/docs
   [junit4]   2> 41370 INFO  (OldIndexDirectoryCleanupThreadForCore-00004_core) [    x:00004_core] o.a.s.c.SolrCore Looking for old index directories to cleanup for core 00004_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/data/
   [junit4]   2> 41370 WARN  (OldIndexDirectoryCleanupThreadForCore-00004_core) [    x:00004_core] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 41371 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 41372 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-005 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41372 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41373 INFO  (OldIndexDirectoryCleanupThreadForCore-00003_core) [    x:00003_core] o.a.s.c.SolrCore Looking for old index directories to cleanup for core 00003_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/data/
   [junit4]   2> 41373 WARN  (OldIndexDirectoryCleanupThreadForCore-00003_core) [    x:00003_core] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00003_core/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 41374 INFO  (OldIndexDirectoryCleanupThreadForCore-00000_core) [    x:00000_core] o.a.s.c.SolrCore Looking for old index directories to cleanup for core 00000_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/data/
   [junit4]   2> 41374 WARN  (OldIndexDirectoryCleanupThreadForCore-00000_core) [    x:00000_core] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 41374 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 41375 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 41376 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41376 INFO  (OldIndexDirectoryCleanupThreadForCore-00006_core) [    x:00006_core] o.a.s.c.SolrCore Looking for old index directories to cleanup for core 00006_core in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/data/
   [junit4]   2> 41376 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 41377 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41378 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41379 WARN  (OldIndexDirectoryCleanupThreadForCore-00006_core) [    x:00006_core] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 41379 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 41380 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 41380 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8168663170157862]
   [junit4]   2> 41382 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41382 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41384 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41384 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41385 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41386 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.RequestHandlers Registered paths: /select,/replication,/admin/threads,/admin/file,/update/csv,/admin/plugins,/get,/admin/properties,/admin/logging,/update/json,/admin/ping,/admin/system,/update,/admin/luke,/config,/admin/segments,/admin/mbeans,/schema,/update/json/docs
   [junit4]   2> 41387 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41387 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 41388 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41388 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41389 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41389 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.RequestHandlers Registered paths: /select,/replication,/admin/threads,/admin/file,/update/csv,/admin/plugins,/get,/admin/properties,/admin/logging,/update/json,/admin/ping,/admin/system,/update,/admin/luke,/config,/admin/segments,/admin/mbeans,/schema,/update/json/docs
   [junit4]   2> 41390 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41390 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 41391 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41391 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41392 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41388 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 41393 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 41393 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8168663170157862]
   [junit4]   2> 41394 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.RequestHandlers Registered paths: /select,/replication,/admin/threads,/admin/file,/update/csv,/admin/plugins,/get,/admin/properties,/admin/logging,/update/json,/admin/ping,/admin/system,/update,/admin/luke,/config,/admin/segments,/admin/mbeans,/schema,/update/json/docs
   [junit4]   2> 41395 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41395 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41395 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 41396 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 41396 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 41396 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8168663170157862]
   [junit4]   2> 41397 INFO  (qtp1120754047-407) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 41397 INFO  (qtp1120754047-407) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00002_core/conf/schema-tiny.xml
   [junit4]   2> 41398 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-005 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41398 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41399 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.s.SolrIndexSearcher Opening Searcher@296f4446[00006_core] main
   [junit4]   2> 41399 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41399 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/conf
   [junit4]   2> 41399 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/conf}
   [junit4]   2> 41400 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00006_core/conf
   [junit4]   2> 41400 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.s.SolrIndexSearcher Opening Searcher@37f381b2[00005_core] main
   [junit4]   2> 41400 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 41400 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 41400 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.CoreContainer Opening transient core 00006_core
   [junit4]   2> 41404 INFO  (qtp1120754047-401) [    x:00006_core] o.a.s.c.CoreContainer registering core: 00006_core
   [junit4]   2> 41392 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 41407 INFO  (qtp1120754047-407) [    ] o.a.s.s.IndexSchema [00002_core] Schema name=tiny
   [junit4]   2> 41407 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 41401 INFO  (searcherExecutor-106-thread-1-processing-x:00006_core) [    x:00006_core] o.a.s.c.SolrCore [00006_core] Registered new searcher Searcher@296f4446[00006_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 41401 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/conf
   [junit4]   2> 41409 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/conf}
   [junit4]   2> 41409 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00005_core/conf
   [junit4]   2> 41409 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 41409 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 41411 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.CoreContainer Opening transient core 00005_core
   [junit4]   2> 41411 INFO  (qtp1120754047-370) [    x:00005_core] o.a.s.c.CoreContainer registering core: 00005_core
   [junit4]   2> 41408 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8168663170157862]
   [junit4]   2> 41401 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 41414 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.RequestHandlers Registered paths: /select,/replication,/admin/threads,/admin/file,/update/csv,/admin/plugins,/get,/admin/properties,/admin/logging,/update/json,/admin/ping,/admin/system,/update,/admin/luke,/config,/admin/segments,/admin/mbeans,/schema,/update/json/docs
   [junit4]   2> 41395 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.s.SolrIndexSearcher Opening Searcher@3ffe5e31[00004_core] main
   [junit4]   2> 41411 INFO  (searcherExecutor-103-thread-1-processing-x:00005_core) [    x:00005_core] o.a.s.c.SolrCore [00005_core] Registered new searcher Searcher@37f381b2[00005_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 41415 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/conf
   [junit4]   2> 41415 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/conf}
   [junit4]   2> 41416 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00004_core/conf
   [junit4]   2> 41416 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 41416 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 41416 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.CoreContainer Opening transient core 00004_core
   [junit4]   2> 41417 INFO  (searcherExecutor-102-thread-1-processing-x:00004_core) [    x:00004_core] o.a.s.c.SolrCore [00004_core] Registered new searcher Searcher@3ffe5e31[00004_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 41417 INFO  (qtp1120754047-399) [    x:00004_core] o.a.s.c.CoreContainer registering core: 00004_core
   [junit4]   2> 41422 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/index-NIOFSDirectory-003 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f3ad072; maxCacheMB=0.4189453125 maxMergeSizeMB=0.134765625)),segFN=segments_1,generation=1}
   [junit4]   2> 41422 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 41422 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.s.SolrIndexSearcher Opening Searcher@9761d26[00000_core] main
   [junit4]   2> 41423 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/conf
   [junit4]   2> 41425 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/conf}
   [junit4]   2> 41425 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_8B4E93F72888A614-001/tempDir-001/00000_core/conf
   [junit4]   2> 41426 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 41426 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 41427 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 41428 INFO  (searcherExecutor-104-thread-1-processing-x:00000_core) [    x:00000_core] o.a.s.c.SolrCore [00000_core] Registered new searcher Searcher@9761d26[00000_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 41428 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.CoreContainer Opening transient core 00000_core
   [junit4]   2> 41428 INFO  (qtp1120754047-372) [    x:00000_core] o.a.s.c.CoreContainer registering core: 00000_core
   [junit4]   2> 41431 INFO  (qtp1120754047-404) [    x:00004_core] o.a.s.c.S.Request [00004_core] webapp=/solr path=/select params={qt=/select&q=*:*&version=2&wt=javabin} hits=0 status=0 QTime=311 
   [junit4]   2> 41432 INFO  (qtp1120754047-411) [    x:00006_core] o.a.s.c.S.Request [00006_core] webapp=/solr path=/select params={qt=/select&q=*:*&version=2&wt=javabin} hits=0 status=0 QTime=139 
   [junit4]   2> 41436 INFO  (qtp1120754047-407) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 41438 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 41439 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 41439 INFO  (qtp1120754047-371) [    x:00003_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByt

[...truncated too long message...]

r.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 5455472 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8B4E93F72888A614]-worker) [n:127.0.0.1:58435__t c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 20, 2015 11:53:52 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=32453, maxDocsPerChunk=292, blockSize=7), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=32453, blockSize=7)), sim=DefaultSimilarity, locale=mk_MK, timezone=Pacific/Truk
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=165870368,total=532152320
   [junit4]   2> NOTE: All tests run in this JVM: [TestEmbeddedSolrServerConstructors, TestSimpleQParserPlugin, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DeleteLastCustomShardedReplicaTest, TestQueryTypes, ZkCLITest, OpenCloseCoreStressTest, TestSchemaVersionResource, TermVectorComponentTest, CSVRequestHandlerTest, UpdateRequestProcessorFactoryTest, TestCustomSort, HdfsRecoverLeaseTest, AnalyticsMergeStrategyTest, TestCustomDocTransformer, CachingDirectoryFactoryTest, TestHighFrequencyDictionaryFactory, TestConfigSets, TestBlendedInfixSuggestions, SSLMigrationTest, TestPivotHelperCode, DocValuesTest, PreAnalyzedFieldTest, TestCursorMarkWithoutUniqueKey, TestFileDictionaryLookup, TestRTimerTree, TestConfigOverlay, HdfsRecoveryZkTest, TestBM25SimilarityFactory, HttpPartitionTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestSearcherReuse, TestRequestStatusCollectionAPI, TestRuleBasedAuthorizationPlugin, UniqFieldsUpdateProcessorFactoryTest, MultiTermTest, TestRTGBase, TestPartialUpdateDeduplication, ZkControllerTest, SolrPluginUtilsTest, TestFastOutputStream, TestConfigSetImmutable, CircularListTest, SolrIndexSplitterTest, TestHashQParserPlugin, SimplePostToolTest, RankQueryTest, PKIAuthenticationIntegrationTest, MinimalSchemaTest, TestLMJelinekMercerSimilarityFactory, TestDistributedGrouping, ChaosMonkeySafeLeaderTest, ExitableDirectoryReaderTest, CollectionsAPIDistributedZkTest, AsyncMigrateRouteKeyTest, TermsComponentTest, DocValuesMissingTest, TestImplicitCoreProperties, TestFieldCollectionResource, DeleteReplicaTest, SoftAutoCommitTest, TestCloudManagedSchemaConcurrent, TestTolerantSearch, URLClassifyProcessorTest, HLLUtilTest, TestCloudSchemaless, OutOfBoxZkACLAndCredentialsProvidersTest, SolrIndexConfigTest, LoggingHandlerTest, VMParamsZkACLAndCredentialsProvidersTest, ShardRoutingCustomTest, FastVectorHighlighterTest, TestMiniSolrCloudCluster, DistributedFacetPivotSmallAdvancedTest, HdfsThreadLeakTest, TestSurroundQueryParser, TestStressVersions, TestDFRSimilarityFactory, SpellPossibilityIteratorTest, ShardRoutingTest, TestReloadAndDeleteDocs, TestHashPartitioner, TestBlobHandler, CollectionStateFormat2Test, TestIntervalFaceting, TestManagedResource, NotRequiredUniqueKeyTest, ConcurrentDeleteAndCreateCollectionTest, MigrateRouteKeyTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, ShardSplitTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [539/539] on J1 in 2024.06s, 1 test, 1 error <<< FAILURES!

[...truncated 16 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites, 2128 tests, 3 errors, 3 failures, 29 ignored (17 assumptions)

Total time: 217 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1038938845 bytes
Compression is 0.0%
Took 3 min 15 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 961 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/961/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=211259, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=211259, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:51007/y/hz: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([CDBE9492836E6DF3]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=4986, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4986, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34657: Could not find collection : awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([CDBE9492836E6DF3]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10447 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/init-core-data-001
   [junit4]   2> 413995 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 413995 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 414774 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 415714 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 416003 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 416016 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 416120 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 416208 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_54408_hdfs____.pn70gp/webapp
   [junit4]   2> 416466 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 417517 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54408
   [junit4]   2> 418610 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 418619 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 418638 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58915_datanode____hogbk1/webapp
   [junit4]   2> 418804 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 419324 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58915
   [junit4]   2> 419450 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 419453 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 419476 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_46523_datanode____.f63kbp/webapp
   [junit4]   2> 419600 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 420433 INFO  (IPC Server handler 3 on 45444) [    ] BlockStateChange BLOCK* processReport: from storage DS-91fba118-7121-4832-9b9c-94d82c2dd86a node DatanodeRegistration(127.0.0.1, datanodeUuid=5a0da851-103b-4afc-a144-361eef7e30d8, infoPort=58915, ipcPort=55518, storageInfo=lv=-56;cid=testClusterID;nsid=1652959030;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 420436 INFO  (IPC Server handler 3 on 45444) [    ] BlockStateChange BLOCK* processReport: from storage DS-90731b45-b861-4534-8636-c1cf0d6511af node DatanodeRegistration(127.0.0.1, datanodeUuid=5a0da851-103b-4afc-a144-361eef7e30d8, infoPort=58915, ipcPort=55518, storageInfo=lv=-56;cid=testClusterID;nsid=1652959030;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 420469 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46523
   [junit4]   2> 420678 INFO  (IPC Server handler 4 on 45444) [    ] BlockStateChange BLOCK* processReport: from storage DS-ad102367-4b78-412d-891d-7c3834cfe50a node DatanodeRegistration(127.0.0.1, datanodeUuid=dea293cf-ecf5-4b72-b11a-65cdc2d88a27, infoPort=46523, ipcPort=47041, storageInfo=lv=-56;cid=testClusterID;nsid=1652959030;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 420681 INFO  (IPC Server handler 4 on 45444) [    ] BlockStateChange BLOCK* processReport: from storage DS-32dfb262-1e28-4353-99b1-c133f8818856 node DatanodeRegistration(127.0.0.1, datanodeUuid=dea293cf-ecf5-4b72-b11a-65cdc2d88a27, infoPort=46523, ipcPort=47041, storageInfo=lv=-56;cid=testClusterID;nsid=1652959030;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 421663 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 421664 INFO  (Thread-940) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 421664 INFO  (Thread-940) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 421764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.ZkTestServer start zk server on port:37848
   [junit4]   2> 421764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 421766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 421769 INFO  (zkCallback-995-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38264127 name:ZooKeeperConnection Watcher:127.0.0.1:37848 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 421769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 421770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 421770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 421779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 421780 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 421782 INFO  (zkCallback-996-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@90d677f name:ZooKeeperConnection Watcher:127.0.0.1:37848/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 421783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 421783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 421783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 421787 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 421789 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 421792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 421794 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 421795 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 421799 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 421799 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 421802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 421803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 421805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 421805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 421808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 421808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 421811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 421811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 421814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 421814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 421817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 421817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 421819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 421820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 421822 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 421823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 421826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 421826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 422151 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores/collection1
   [junit4]   2> 422153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 422155 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@51400e5f{/,null,AVAILABLE}
   [junit4]   2> 422156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@446fa8a5{HTTP/1.1}{127.0.0.1:34657}
   [junit4]   2> 422156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.Server Started @425305ms
   [junit4]   2> 422156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34657, solr.data.dir=hdfs://localhost:45444/hdfs__localhost_45444__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores, hostContext=/}
   [junit4]   2> 422156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 422157 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/'
   [junit4]   2> 422172 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 422173 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 422175 INFO  (zkCallback-997-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d6745a4 name:ZooKeeperConnection Watcher:127.0.0.1:37848/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 422175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 422175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 422177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 422178 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/solr.xml
   [junit4]   2> 422193 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores
   [junit4]   2> 422193 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoreContainer New CoreContainer 1110012043
   [junit4]   2> 422193 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/]
   [junit4]   2> 422193 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/lib
   [junit4]   2> 422194 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/lib).
   [junit4]   2> 422202 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 422204 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 422205 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 422205 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 422205 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37848/solr
   [junit4]   2> 422205 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 422205 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 422206 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 422209 INFO  (zkCallback-999-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@543ca941 name:ZooKeeperConnection Watcher:127.0.0.1:37848 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 422210 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 422211 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 422218 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 422219 INFO  (zkCallback-1000-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ba2c090 name:ZooKeeperConnection Watcher:127.0.0.1:37848/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 422219 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 422221 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 422224 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 422228 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 422231 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 422234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 422261 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 422265 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 422267 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 422269 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 422270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34657_
   [junit4]   2> 422271 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34657_
   [junit4]   2> 422273 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 422275 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 422278 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 422279 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94550205800972292-127.0.0.1:34657_-n_0000000000
   [junit4]   2> 422280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34657_
   [junit4]   2> 422280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 422282 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer Overseer (id=94550205800972292-127.0.0.1:34657_-n_0000000000) starting
   [junit4]   2> 422285 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 422294 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 422295 INFO  (OverseerCollectionConfigSetProcessor-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 422295 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 422296 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 422323 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 422323 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 422325 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores
   [junit4]   2> 422326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores/collection1, coreNodeName=, configSetProperties=configsetprops.json, transient=false, dataDir=data/, loadOnStartup=true, name=collection1, shard=, schema=schema.xml, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores/collection1/}
   [junit4]   2> 422326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores/collection1/
   [junit4]   2> 422326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 422327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 422327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:34657_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 422328 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 422328 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 422331 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:34657",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:34657_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 422332 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 422332 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:34657",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:34657_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 422333 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 422333 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 422336 INFO  (zkCallback-1000-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 423333 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 423333 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 423334 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 423334 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 423335 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 423335 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores/collection1/'
   [junit4]   2> 423352 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 423361 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 423376 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 423406 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 423407 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 423427 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 423718 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 423726 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 423728 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 423738 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 423742 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 423746 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 423746 WARN  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 423746 WARN  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 423747 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 423748 WARN  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 423748 WARN  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 423749 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 423749 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 423749 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 423760 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45444/solr_hdfs_home
   [junit4]   2> 423760 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 423760 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 423760 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@135a4815
   [junit4]   2> 423761 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 423848 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 423849 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 423855 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 423886 WARN  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:45444/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 423886 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 423909 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 423909 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 423910 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.7333984375, noCFSRatio=0.13717099018503146]
   [junit4]   2> 424268 INFO  (IPC Server handler 5 on 45444) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60270 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-90731b45-b861-4534-8636-c1cf0d6511af:NORMAL:127.0.0.1:56938|RBW], ReplicaUnderConstruction[[DISK]DS-32dfb262-1e28-4353-99b1-c133f8818856:NORMAL:127.0.0.1:60270|RBW]]} size 71
   [junit4]   2> 424270 INFO  (IPC Server handler 3 on 45444) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56938 is added to blk_1073741825_1001 size 71
   [junit4]   2> 424689 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b770ca6a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e471e68; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 424689 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 424694 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:45444/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 424700 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 424700 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 424701 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 424701 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 424701 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 424701 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 424701 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 424702 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 424702 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 424705 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 424707 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 424708 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 424710 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 424714 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /config,standard,/admin/logging,/admin/segments,/admin/file,/update/csv,/update/json/docs,/admin/mbeans,/update,/admin/properties,/get,/admin/threads,/schema,/admin/ping,/admin/system,/admin/luke,/admin/plugins,/update/json,/replication
   [junit4]   2> 424717 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 424739 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 424741 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 424763 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 424763 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 424766 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@75bed208
   [junit4]   2> 424847 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b770ca6a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e471e68; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 424847 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 424848 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2f266fcd[collection1] main
   [junit4]   2> 424848 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 424849 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 424849 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 424850 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 424850 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 424851 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 424851 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 424852 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 424852 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 424852 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 424854 INFO  (searcherExecutor-842-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2f266fcd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 424861 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 424861 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 424862 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 424862 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1512803296165756928
   [junit4]   2> 424862 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1512803296165756928
   [junit4]   2> 424862 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 424863 INFO  (coreLoadExecutor-841-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 424863 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34657 collection:control_collection shard:shard1
   [junit4]   2> 424864 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 424870 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94550205800972292-core_node1-n_0000000000
   [junit4]   2> 424871 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 424873 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 424874 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 424874 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34657/collection1/
   [junit4]   2> 424874 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 424874 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 424875 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34657/collection1/ has no replicas
   [junit4]   2> 424876 INFO  (zkCallback-1000-thread-1-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 424876 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34657/collection1/ shard1
   [junit4]   2> 424878 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 424882 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 424885 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:34657",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 424988 INFO  (zkCallback-1000-thread-2-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 425035 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:34657/collection1/ and leader is http://127.0.0.1:34657/collection1/
   [junit4]   2> 425036 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34657
   [junit4]   2> 425036 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 425036 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 425036 INFO  (coreZkRegister-835-thread-1-processing-x:collection1 n:127.0.0.1:34657_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:34657_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 425039 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34657",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:34657_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 425040 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34657",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:34657_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 425143 INFO  (zkCallback-1000-thread-2-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 425330 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 425331 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 425333 INFO  (zkCallback-1002-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@d9193a name:ZooKeeperConnection Watcher:127.0.0.1:37848/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 425333 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 425333 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 425334 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 425337 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 425338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 425338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 425338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 425340 INFO  (zkCallback-1003-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@940e2d5 name:ZooKeeperConnection Watcher:127.0.0.1:37848/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 425341 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 425341 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 425344 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 425345 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 425448 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 425695 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/cores/collection1
   [junit4]   2> 425696 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001
   [junit4]   2> 425697 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 425699 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36978ca1{/,null,AVAILABLE}
   [junit4]   2> 425700 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1791ba07{HTTP/1.1}{127.0.0.1:38585}
   [junit4]   2> 425700 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.e.j.s.Server Started @428849ms
   [junit4]   2> 425700 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:45444/hdfs__localhost_45444__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001_tempDir-002_jetty1, hostContext=/, solrconfig=solrconfig.xml, hostPort=38585, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/cores}
   [junit4]   2> 425701 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 425701 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/'
   [junit4]   2> 425719 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 425720 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 425721 INFO  (zkCallback-1004-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f31e1e name:ZooKeeperConnection Watcher:127.0.0.1:37848/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 425721 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 425722 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 425723 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 425724 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/solr.xml
   [junit4]   2> 425744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/cores
   [junit4]   2> 425744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoreContainer New CoreContainer 127358021
   [junit4]   2> 425744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/]
   [junit4]   2> 425745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/lib
   [junit4]   2> 425745 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/lib).
   [junit4]   2> 425756 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 425757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 425758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 425758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 425758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37848/solr
   [junit4]   2> 425758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 425758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 425761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 425761 INFO  (zkCallback-1006-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@437a2eb5 name:ZooKeeperConnection Watcher:127.0.0.1:37848 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 425762 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 425762 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 425764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 425766 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@609afa78 name:ZooKeeperConnection Watcher:127.0.0.1:37848/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 425766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 425774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 426783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38585_
   [junit4]   2> 426785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38585_
   [junit4]   2> 426789 INFO  (zkCallback-1000-thread-2-processing-n:127.0.0.1:34657_) [n:127.0.0.1:34657_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 426789 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 426789 INFO  (zkCallback-1002-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 426792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 426793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94550205800972297-127.0.0.1:38585_-n_0000000001
   [junit4]   2> 426796 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94550205800972292-127.0.0.1:34657_-n_0000000000 to know if I could be the leader
   [junit4]   2> 426814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 426814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 426817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/cores
   [junit4]   2> 426818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/cores/collection1, loadOnStartup=true, name=collection1, config=solrconfig.xml, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/cores/collection1/, configSetProperties=configsetprops.json, transient=false, coreNodeName=, dataDir=data/, schema=schema.xml}
   [junit4]   2> 426819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CDBE9492836E6DF3-001/shard-1-001/cores/collection1/
   [junit4]   2> 426819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 426821 INFO  (coreLoadExecutor-852-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 426821 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 426822 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CDBE9492836E6DF3]) [n:127.0.0.1:38585_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 426822 INFO  (coreLoadExecutor-852-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 426825 INFO  (coreLoadExecutor-852-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 426825 INFO  (coreLoadExecutor-852-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 426826 INFO  (coreLoadExecutor-852-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 426826 INFO  (coreLoadExecutor-852-thread-1-processing-n:127.0.0.1:38585_) [n:127.0.0.1:38585_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 426826 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:38585",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:38585_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"} current state version: 4
   [junit4]   2> 426827 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:38585",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:38585_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"}
   [junit4]   2> 426827 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 426828 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 426932 INFO  (OverseerStateUpdate-94550205800972292-127.0.0.1:34657_-n_0000000000) [n:127.0.0.1:34657_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version

[...truncated too long message...]

olrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=211262, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:51007/y/hz: Could not find collection : awholynewstresscollection_collection4_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([CDBE9492836E6DF3]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=211261, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:51007/y/hz: Could not find collection : awholynewstresscollection_collection3_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([CDBE9492836E6DF3]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=211258, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:39411/y/hz: Could not find collection : awholynewstresscollection_collection0_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([CDBE9492836E6DF3]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]   2> 2486140 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[CDBE9492836E6DF3]-worker) [n:127.0.0.1:57544_y%2Fhz c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=11313, maxDocsPerChunk=951, blockSize=297), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=11313, blockSize=297)), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=sk, timezone=Africa/Maseru
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=95476432,total=520093696
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolrDeletionPolicy1, DebugComponentTest, SolrCoreCheckLockOnStartupTest, QueryParsingTest, ClusterStateTest, AddBlockUpdateTest, BasicDistributedZk2Test, DirectSolrConnectionTest, DocValuesMultiTest, TestDefaultSearchFieldResource, BadIndexSchemaTest, TestExceedMaxTermLength, CursorMarkTest, AsyncCallRequestStatusResponseTest, TestDynamicFieldCollectionResource, TermVectorComponentDistributedTest, DistribJoinFromCollectionTest, ActionThrottleTest, TestCSVResponseWriter, BJQParserTest, LeaderFailoverAfterPartitionTest, LeaderElectionTest, TestSolrQueryParserDefaultOperatorResource, FacetPivotSmallTest, TestRemoveLastDynamicCopyField, TestSweetSpotSimilarityFactory, SolrTestCaseJ4Test, CoreAdminHandlerTest, TestElisionMultitermQuery, TestRecovery, TestLeaderElectionZkExpiry, TestSimpleTrackingShardHandler, TestSolrQueryParserResource, TestAnalyzeInfixSuggestions, TestDistributedStatsComponentCardinality, DistributedFacetPivotLongTailTest, RecoveryZkTest, AutoCommitTest, BitVectorTest, CoreAdminCreateDiscoverTest, HdfsLockFactoryTest, SuggesterWFSTTest, TestCollapseQParserPlugin, TestSolrConfigHandlerCloud, TestDownShardTolerantSearch, HardAutoCommitTest, TestFieldTypeResource, TestDynamicLoading, HdfsUnloadDistributedZkTest, CurrencyFieldXmlFileTest, SampleTest, TestSchemaManager, TestIndexingPerformance, ScriptEngineTest, TestSmileRequest, UpdateParamsTest, TestFieldResource, DistributedExpandComponentTest, TestStressRecovery, TestExtendedDismaxParser, OverseerCollectionConfigSetProcessorTest, HdfsBasicDistributedZkTest, TestJettySolrRunner, CollectionReloadTest, CloudMLTQParserTest, TestAddFieldRealTimeGet, SystemInfoHandlerTest, FullSolrCloudDistribCmdsTest, SuggesterTSTTest, RAMDirectoryFactoryTest, ZkStateWriterTest, SimpleFacetsTest, TestZkChroot, SpatialHeatmapFacetsTest, TestFastWriter, StatelessScriptUpdateProcessorFactoryTest, TestSystemIdResolver, SecurityConfHandlerTest, TestFuzzyAnalyzedSuggestions, LukeRequestHandlerTest, TestShortCircuitedRequests, TestBulkSchemaAPI, TestClassNameShortening, TestMinMaxOnMultiValuedField, PrimitiveFieldTypeTest, TestJsonRequest, TestJsonFacets, SmileWriterTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [272/539] on J2 in 416.93s, 1 test, 1 error <<< FAILURES!

[...truncated 868 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites, 2128 tests, 2 errors, 29 ignored (17 assumptions)

Total time: 192 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1174381448 bytes
Compression is 0.0%
Took 3 min 12 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 960 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/960/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=107732, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=107732, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33032/ah/tv: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([DEE4CC207C9280F7]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11548 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/init-core-data-001
   [junit4]   2> 2977212 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 2977212 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ah/tv
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2977253 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2977261 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2977264 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2977280 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_39771_hdfs____l1jcnp/webapp
   [junit4]   2> 2977402 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2977898 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39771
   [junit4]   2> 2977959 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2977960 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2977974 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_60842_datanode____148s6b/webapp
   [junit4]   2> 2978093 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2978592 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60842
   [junit4]   2> 2978624 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2978626 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2978638 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36416_datanode____2co8ot/webapp
   [junit4]   2> 2978669 INFO  (IPC Server handler 3 on 60399) [    ] BlockStateChange BLOCK* processReport: from storage DS-c8821c1e-6334-4242-afc5-f75f6be3a940 node DatanodeRegistration(127.0.0.1, datanodeUuid=029fdf57-708a-4c35-b444-8bfb28d74688, infoPort=60842, ipcPort=40552, storageInfo=lv=-56;cid=testClusterID;nsid=803210750;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2978670 INFO  (IPC Server handler 3 on 60399) [    ] BlockStateChange BLOCK* processReport: from storage DS-ac019655-08f0-455d-a219-ec4476669384 node DatanodeRegistration(127.0.0.1, datanodeUuid=029fdf57-708a-4c35-b444-8bfb28d74688, infoPort=60842, ipcPort=40552, storageInfo=lv=-56;cid=testClusterID;nsid=803210750;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2978762 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2979107 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36416
   [junit4]   2> 2979181 INFO  (IPC Server handler 4 on 60399) [    ] BlockStateChange BLOCK* processReport: from storage DS-187d403c-520c-4db5-bad8-1f2967b8c6c8 node DatanodeRegistration(127.0.0.1, datanodeUuid=5d270128-c669-4453-9047-217dd4e39dcb, infoPort=36416, ipcPort=48296, storageInfo=lv=-56;cid=testClusterID;nsid=803210750;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 2979182 INFO  (IPC Server handler 4 on 60399) [    ] BlockStateChange BLOCK* processReport: from storage DS-7b220d8f-f2be-4fe5-97c2-2a12437170c3 node DatanodeRegistration(127.0.0.1, datanodeUuid=5d270128-c669-4453-9047-217dd4e39dcb, infoPort=36416, ipcPort=48296, storageInfo=lv=-56;cid=testClusterID;nsid=803210750;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2979295 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2979295 INFO  (Thread-91401) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2979296 INFO  (Thread-91401) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2979396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.ZkTestServer start zk server on port:57695
   [junit4]   2> 2979396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2979397 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2979400 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a60f616 name:ZooKeeperConnection Watcher:127.0.0.1:57695 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2979400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2979400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2979400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2979403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2979404 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2979405 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@664f883c name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2979405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2979406 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2979406 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2979408 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2979410 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2979412 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2979413 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2979414 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2979417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2979417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2979419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2979419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2979421 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2979421 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2979423 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2979423 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2979425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2979425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2979427 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2979427 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2979429 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2979429 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2979431 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2979431 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2979432 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2979433 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2979436 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2979436 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2979438 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2979439 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2979440 INFO  (zkCallback-3075-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35e2fa3e name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2979440 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2979440 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2979442 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14fe46206090002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2979745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores/collection1
   [junit4]   2> 2979747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2979748 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@168e3df5{/ah/tv,null,AVAILABLE}
   [junit4]   2> 2979751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.ServerConnector Started ServerConnector@13b75d96{SSL-http/1.1}{127.0.0.1:39226}
   [junit4]   2> 2979751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.Server Started @2982977ms
   [junit4]   2> 2979751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ah/tv, hostPort=39226, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores, solr.data.dir=hdfs://localhost:60399/hdfs__localhost_60399__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001_tempDir-002_control_data}
   [junit4]   2> 2979752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 2979752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/'
   [junit4]   2> 2979768 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2979769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2979770 INFO  (zkCallback-3076-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66fcd530 name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2979770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2979771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2979772 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2979772 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/solr.xml
   [junit4]   2> 2979786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores
   [junit4]   2> 2979786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoreContainer New CoreContainer 1286102102
   [junit4]   2> 2979786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/]
   [junit4]   2> 2979787 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/lib
   [junit4]   2> 2979787 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/lib).
   [junit4]   2> 2979794 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2979797 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2979798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2979798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2979798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57695/solr
   [junit4]   2> 2979798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2979798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2979799 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2979800 INFO  (zkCallback-3078-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56fb0aee name:ZooKeeperConnection Watcher:127.0.0.1:57695 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2979800 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2979801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2979802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2979803 INFO  (zkCallback-3079-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4cbda36 name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2979803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2979804 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2979806 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2979809 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2979811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2979813 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2979818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2979820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2979823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2979825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2979827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39226_ah%2Ftv
   [junit4]   2> 2979828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39226_ah%2Ftv
   [junit4]   2> 2979830 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2979831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2979833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2979833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000
   [junit4]   2> 2979834 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39226_ah%2Ftv
   [junit4]   2> 2979834 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2979835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer Overseer (id=94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) starting
   [junit4]   2> 2979837 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2979843 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2979844 INFO  (OverseerCollectionConfigSetProcessor-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2979844 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2979844 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2979856 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2979856 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2979857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores
   [junit4]   2> 2979858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores/collection1/, schema=schema.xml, loadOnStartup=true, dataDir=data/, config=solrconfig.xml, coreNodeName=, shard=, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores/collection1, name=collection1, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 2979858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores/collection1/
   [junit4]   2> 2979858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2979859 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2979859 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2979860 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2979860 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2979861 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39226_ah%2Ftv",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39226/ah/tv",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 2979862 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2979862 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:39226_ah%2Ftv",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39226/ah/tv",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 2979862 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2979863 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2979864 INFO  (zkCallback-3079-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2980862 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2980863 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2980863 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2980863 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2980864 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2980864 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores/collection1/'
   [junit4]   2> 2980879 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2980885 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2980898 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2980926 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2980927 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2980934 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2981207 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2981215 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2981217 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2981226 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2981228 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2981231 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2981232 WARN  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2981232 WARN  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2981232 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2981233 WARN  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2981233 WARN  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2981234 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2981234 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2981235 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 2981235 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:60399/solr_hdfs_home
   [junit4]   2> 2981235 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2981235 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2981235 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@748ca89c
   [junit4]   2> 2981236 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2981250 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 2981251 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2981252 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 2981261 WARN  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:60399/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2981261 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2981273 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 2981273 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2981273 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4a80a2b4
   [junit4]   2> 2981286 INFO  (IPC Server handler 0 on 60399) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43679 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-c8821c1e-6334-4242-afc5-f75f6be3a940:NORMAL:127.0.0.1:41775|RBW], ReplicaUnderConstruction[[DISK]DS-187d403c-520c-4db5-bad8-1f2967b8c6c8:NORMAL:127.0.0.1:43679|RBW]]} size 0
   [junit4]   2> 2981288 INFO  (IPC Server handler 1 on 60399) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41775 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-187d403c-520c-4db5-bad8-1f2967b8c6c8:NORMAL:127.0.0.1:43679|RBW], ReplicaUnderConstruction[[DISK]DS-ac019655-08f0-455d-a219-ec4476669384:NORMAL:127.0.0.1:41775|FINALIZED]]} size 0
   [junit4]   2> 2981291 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92d8b09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4f948596; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2981291 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2981292 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:60399/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 2981296 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2981297 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2981297 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2981297 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2981297 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2981297 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2981298 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2981298 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2981298 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2981301 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2981302 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2981303 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2981305 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2981308 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/admin/logging,/admin/luke,/update/csv,/admin/mbeans,/config,/admin/properties,/admin/ping,/admin/threads,/update/json,/update/json/docs,/admin/segments,/replication,/get,/admin/system,/admin/file,/schema,/update,standard
   [junit4]   2> 2981310 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2981310 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2981311 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 2981321 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2981321 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2981322 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2981327 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92d8b09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4f948596; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2981327 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2981327 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@30b0b7f7[collection1] main
   [junit4]   2> 2981327 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2981328 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2981328 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2981329 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2981329 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2981329 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2981329 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2981330 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2981330 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2981330 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2981331 INFO  (searcherExecutor-6355-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@30b0b7f7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2981333 WARN  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:52219/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:52219 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 2981333 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2981333 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 2981333 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2981333 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1512723631850913792
   [junit4]   2> 2981333 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1512723631850913792
   [junit4]   2> 2981333 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2981334 INFO  (coreLoadExecutor-6354-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2981334 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:39226/ah/tv collection:control_collection shard:shard1
   [junit4]   2> 2981335 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2981339 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94545226857250821-core_node1-n_0000000000
   [junit4]   2> 2981339 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2981341 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2981341 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2981341 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:39226/ah/tv/collection1/
   [junit4]   2> 2981341 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2981341 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2981341 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:39226/ah/tv/collection1/ has no replicas
   [junit4]   2> 2981342 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:39226/ah/tv/collection1/ shard1
   [junit4]   2> 2981343 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2981347 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2981349 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:39226/ah/tv",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2981363 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2981363 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2981365 INFO  (zkCallback-3081-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5382c65 name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2981365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2981365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2981365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2981368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2981368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2981368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2981368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2981370 INFO  (zkCallback-3082-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@41f73fb2 name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2981370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2981370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2981372 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 2981373 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2981373 INFO  (zkCallback-3081-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2981373 INFO  (zkCallback-3079-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2981399 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:39226/ah/tv/collection1/ and leader is https://127.0.0.1:39226/ah/tv/collection1/
   [junit4]   2> 2981400 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:39226/ah/tv
   [junit4]   2> 2981400 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2981400 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2981400 INFO  (coreZkRegister-6348-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:39226_ah%2Ftv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2981402 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39226_ah%2Ftv",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"https://127.0.0.1:39226/ah/tv",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection"} current state version: 2
   [junit4]   2> 2981402 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:39226_ah%2Ftv",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"https://127.0.0.1:39226/ah/tv",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 2981404 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2981508 INFO  (zkCallback-3079-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2981508 INFO  (zkCallback-3081-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2981641 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/cores/collection1
   [junit4]   2> 2981642 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001
   [junit4]   2> 2981643 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2981645 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b3b3edf{/ah/tv,null,AVAILABLE}
   [junit4]   2> 2981647 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7da45d96{SSL-http/1.1}{127.0.0.1:33032}
   [junit4]   2> 2981648 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.e.j.s.Server Started @2984874ms
   [junit4]   2> 2981648 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=33032, solr.data.dir=hdfs://localhost:60399/hdfs__localhost_60399__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/cores, hostContext=/ah/tv}
   [junit4]   2> 2981648 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 2981648 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/'
   [junit4]   2> 2981665 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2981666 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2981667 INFO  (zkCallback-3083-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@548f056e name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2981668 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2981670 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2981671 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2981672 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/solr.xml
   [junit4]   2> 2981686 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/cores
   [junit4]   2> 2981686 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoreContainer New CoreContainer 2109051372
   [junit4]   2> 2981686 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/]
   [junit4]   2> 2981686 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/lib
   [junit4]   2> 2981686 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/lib).
   [junit4]   2> 2981706 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2981708 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2981709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2981709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2981709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57695/solr
   [junit4]   2> 2981710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2981710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2981710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2981711 INFO  (zkCallback-3085-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a4a737a name:ZooKeeperConnection Watcher:127.0.0.1:57695 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2981712 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2981712 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2981714 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2981714 INFO  (zkCallback-3086-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a008040 name:ZooKeeperConnection Watcher:127.0.0.1:57695/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2981715 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2981720 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2982750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33032_ah%2Ftv
   [junit4]   2> 2982751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33032_ah%2Ftv
   [junit4]   2> 2982752 INFO  (zkCallback-3081-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2982752 INFO  (zkCallback-3086-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2982752 INFO  (zkCallback-3079-thread-1-processing-n:127.0.0.1:39226_ah%2Ftv) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2982753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2982754 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94545226857250826-127.0.0.1:33032_ah%2Ftv-n_0000000001
   [junit4]   2> 2982755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000 to know if I could be the leader
   [junit4]   2> 2982765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2982765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2982766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/cores
   [junit4]   2> 2982767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/cores/collection1/, loadOnStartup=true, transient=false, config=solrconfig.xml, configSetProperties=configsetprops.json, schema=schema.xml, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/cores/collection1, collection=collection1, shard=, dataDir=data/, name=collection1}
   [junit4]   2> 2982767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_DEE4CC207C9280F7-001/shard-1-001/cores/collection1/
   [junit4]   2> 2982767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2982769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2982769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[DEE4CC207C9280F7]) [n:127.0.0.1:33032_ah%2Ftv    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2982769 INFO  (coreLoadExecutor-6365-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2982769 INFO  (coreLoadExecutor-6365-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2982771 INFO  (coreLoadExecutor-6365-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2982771 INFO  (coreLoadExecutor-6365-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 2982771 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:33032_ah%2Ftv",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33032/ah/tv",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"} current state version: 3
   [junit4]   2> 2982771 INFO  (coreLoadExecutor-6365-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 2982772 INFO  (coreLoadExecutor-6365-thread-1-processing-n:127.0.0.1:33032_ah%2Ftv) [n:127.0.0.1:33032_ah%2Ftv c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2982772 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:33032_ah%2Ftv",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33032/ah/tv",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"}
   [junit4]   2> 2982772 INFO  (OverseerStateUpdate-94545226857250821-127.0.0.1:39226_ah%2Ftv-n_0000000000) [n:127.0.0.1:39226_ah%2Ftv    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 2982772 INFO  (OverseerStateUpdate-94545226857250821-127.0.0

[...truncated too long message...]

nterceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4339386 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[DEE4CC207C9280F7]-worker) [n:127.0.0.1:54102_ah%2Ftv c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 19, 2015 3:17:14 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene53, sim=DefaultSimilarity, locale=hr_HR, timezone=America/Puerto_Rico
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=226487808,total=530579456
   [junit4]   2> NOTE: All tests run in this JVM: [SampleTest, TestLRUCache, TestExtendedDismaxParser, BJQParserTest, StatsComponentTest, TestFastWriter, LukeRequestHandlerTest, TestAuthorizationFramework, PrimitiveFieldTypeTest, TestConfigSetsAPIExclusivity, TestDynamicFieldCollectionResource, MergeStrategyTest, SimpleFacetsTest, TestWriterPerf, TestPostingsSolrHighlighter, DistribJoinFromCollectionTest, TestSolrQueryParserResource, ZkNodePropsTest, HdfsBasicDistributedZkTest, TestAddFieldRealTimeGet, TestShardHandlerFactory, DistributedSuggestComponentTest, TestRecovery, TestSolrDynamicMBean, TestPerFieldSimilarity, SearchHandlerTest, HdfsUnloadDistributedZkTest, TestDefaultStatsCache, TestCharFilters, TestSolrConfigHandlerCloud, TestSearchPerf, DirectSolrConnectionTest, TestFuzzyAnalyzedSuggestions, SuggesterWFSTTest, TestSimpleTrackingShardHandler, DistributedQueryComponentCustomSortTest, TestManagedSchemaFieldResource, HdfsDirectoryFactoryTest, TestCoreDiscovery, SolrTestCaseJ4Test, TestSystemIdResolver, RequestHandlersTest, DistribDocExpirationUpdateProcessorTest, CloudMLTQParserTest, TestExceedMaxTermLength, TestNRTOpen, HdfsLockFactoryTest, PreAnalyzedUpdateProcessorTest, SmileWriterTest, CurrencyFieldXmlFileTest, TestSolrQueryParser, TestSortingResponseWriter, UUIDFieldTest, RAMDirectoryFactoryTest, TestRemoveLastDynamicCopyField, OverseerTest, EnumFieldTest, HardAutoCommitTest, TestXIncludeConfig, TestJsonRequest, BasicDistributedZk2Test, TestFieldTypeResource, TestZkChroot, TermVectorComponentDistributedTest, BinaryUpdateRequestHandlerTest, BasicAuthIntegrationTest, BitVectorTest, TestLeaderInitiatedRecoveryThread, TestIndexingPerformance, HighlighterConfigTest, TestSolrCloudWithKerberosAlt, TestClassNameShortening, TestCollapseQParserPlugin, TestGroupingSearch, RecoveryZkTest, UnloadDistributedZkTest, BadComponentTest, TestFastLRUCache, DirectUpdateHandlerOptimizeTest, TestStressRecovery, ZkStateWriterTest, SpatialHeatmapFacetsTest, TestSolrQueryParserDefaultOperatorResource, TestElisionMultitermQuery, SolrCoreCheckLockOnStartupTest, DirectUpdateHandlerTest, SparseHLLTest, CollectionsAPIDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest, TestRandomFaceting, LeaderElectionTest, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, LeaderFailoverAfterPartitionTest, SharedFSAutoReplicaFailoverTest, TestConfigSetsAPI, TestCryptoKeys, TestDownShardTolerantSearch, TestExclusionRuleCollectionAccess, TestRandomRequestDistribution, TestRequestStatusCollectionAPI, VMParamsZkACLAndCredentialsProvidersTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [539/539] on J2 in 1372.06s, 1 test, 1 error <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 539 suites, 2128 tests, 1 error, 29 ignored (17 assumptions)

Total time: 192 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 886102227 bytes
Compression is 0.0%
Took 2 min 43 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 959 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/959/

No tests ran.

Build Log:
[...truncated 161 lines...]
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x
org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:777)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775)
	... 35 more
Caused by: svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:514)
	... 35 more
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:146)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:73)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:280)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:451)
	... 35 more
ERROR: Subversion update failed
java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x at hudson.remoting.Channel@616b30a5:lucene
	at hudson.FilePath.act(FilePath.java:916)
	at hudson.FilePath.act(FilePath.java:893)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:936)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:871)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:615)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:524)
	at hudson.model.Run.execute(Run.java:1706)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:211)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.scm.subversion.UpdaterException: failed to perform svn update
	... 14 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	... 13 more
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:777)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775)
	... 35 more
Caused by: svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:514)
	... 35 more
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:146)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:73)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:280)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:451)
	... 35 more
Retrying after 10 seconds
Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/.
Updating http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x at revision '2015-09-17T14:48:58.359 -0400'
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x
org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:777)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775)
	... 35 more
Caused by: svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:514)
	... 35 more
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:146)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:73)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:280)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:451)
	... 35 more
ERROR: Subversion update failed
java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x at hudson.remoting.Channel@616b30a5:lucene
	at hudson.FilePath.act(FilePath.java:916)
	at hudson.FilePath.act(FilePath.java:893)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:936)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:871)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:615)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:524)
	at hudson.model.Run.execute(Run.java:1706)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:211)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.scm.subversion.UpdaterException: failed to perform svn update
	... 14 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	... 13 more
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:777)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775)
	... 35 more
Caused by: svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:514)
	... 35 more
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:146)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:73)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:280)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:451)
	... 35 more
Retrying after 10 seconds
Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/.
Updating http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x at revision '2015-09-17T14:48:58.359 -0400'
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x
org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:777)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775)
	... 35 more
Caused by: svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:514)
	... 35 more
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:146)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:73)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:280)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:451)
	... 35 more
ERROR: Subversion update failed
java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x at hudson.remoting.Channel@616b30a5:lucene
	at hudson.FilePath.act(FilePath.java:916)
	at hudson.FilePath.act(FilePath.java:893)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:936)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:871)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:615)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:524)
	at hudson.model.Run.execute(Run.java:1706)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:211)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.scm.subversion.UpdaterException: failed to perform svn update
	... 14 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	... 13 more
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:777)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775)
	... 35 more
Caused by: svn: E175002: timed out waiting for server
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:514)
	... 35 more
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:146)
	at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:73)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:280)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:451)
	... 35 more
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 0.1 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 958 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/958/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=75073, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=75073, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56516/ckb/p: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([428DD5848704966F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11404 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150917_054621_217.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid27542.hprof ...
   [junit4] Heap dump file created [658512451 bytes in 13.746 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150917_054621_217.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J2: EOF ----

[...truncated 43 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/init-core-data-001
   [junit4]   2> 3999514 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 3999514 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ckb/p
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3999561 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3999571 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3999575 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3999591 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_44299_hdfs____.khwa2s/webapp
   [junit4]   2> 3999725 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4000247 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44299
   [junit4]   2> 4000317 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4000320 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4000336 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41501_datanode____.hk1zcw/webapp
   [junit4]   2> 4000468 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4000938 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41501
   [junit4]   2> 4000980 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4000982 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4001005 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47395_datanode____cpgyb/webapp
   [junit4]   2> 4001061 INFO  (IPC Server handler 3 on 43965) [    ] BlockStateChange BLOCK* processReport: from storage DS-4840cdd7-b662-4b12-8485-d5a9b0981daf node DatanodeRegistration(127.0.0.1, datanodeUuid=0f87ae04-5d01-436e-ba1c-b20401995f75, infoPort=41501, ipcPort=43208, storageInfo=lv=-56;cid=testClusterID;nsid=542165791;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 4001062 INFO  (IPC Server handler 3 on 43965) [    ] BlockStateChange BLOCK* processReport: from storage DS-929254c3-073f-4cad-ab4d-7d714322c01b node DatanodeRegistration(127.0.0.1, datanodeUuid=0f87ae04-5d01-436e-ba1c-b20401995f75, infoPort=41501, ipcPort=43208, storageInfo=lv=-56;cid=testClusterID;nsid=542165791;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4001172 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4001671 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[428DD5848704966F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47395
   [junit4]   2> 4001752 INFO  (IPC Server handler 6 on 43965) [    ] BlockStateChange BLOCK* processReport: from storage DS-6e64c2f4-188b-4e39-884a-2a51386b8e1c node DatanodeRegistration(127.0.0.1, datanodeUuid=024e9952-bd36-425b-a5fe-b8b32fd8a2a6, infoPort=47395, ipcPort=59239, storageInfo=lv=-56;cid=testClusterID;nsid=542165791;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 4001753 INFO  (IPC Server handler 6 on 43965) [    ] BlockStateChange BLOCK* processReport: from storage DS-5eb5d3af-17a5-4720-8323-eaa0e27a5d03 node DatanodeRegistration(127.0.0.1, datanodeUuid=024e9952-bd36-425b-a5fe-b8b32fd8a2a6, infoPort=47395, ipcPort=59239, storageInfo=lv=-56;cid=testClusterID;nsid=542165791;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4001850 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4001850 INFO  (Thread-63154) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4001851 INFO  (Thread-63154) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4001950 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.ZkTestServer start zk server on port:55147
   [junit4]   2> 4001951 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4001952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4001955 INFO  (zkCallback-2284-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c688a85 name:ZooKeeperConnection Watcher:127.0.0.1:55147 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4001955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4001955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4001956 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4001959 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4001960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4001961 INFO  (zkCallback-2285-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6980045a name:ZooKeeperConnection Watcher:127.0.0.1:55147/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4001961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4001962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4001962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 4001964 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 4001966 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 4001967 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 4001969 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4001970 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4001973 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4001973 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 4001976 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4001976 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4001978 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4001979 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4001981 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4001981 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4001983 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4001983 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 4001985 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4001985 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4001987 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4001987 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4001989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4001989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4001991 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4001992 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4001993 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4001994 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4002338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores/collection1
   [junit4]   2> 4002340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4002343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2eeb5349{/ckb/p,null,AVAILABLE}
   [junit4]   2> 4002343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3199e32c{HTTP/1.1}{127.0.0.1:44227}
   [junit4]   2> 4002343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.Server Started @4005517ms
   [junit4]   2> 4002344 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ckb/p, hostPort=44227, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores, solr.data.dir=hdfs://localhost:43965/hdfs__localhost_43965__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001_tempDir-002_control_data}
   [junit4]   2> 4002344 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 4002344 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/'
   [junit4]   2> 4002365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4002366 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4002368 INFO  (zkCallback-2286-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c80370 name:ZooKeeperConnection Watcher:127.0.0.1:55147/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4002368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4002368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4002370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4002370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/solr.xml
   [junit4]   2> 4002388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores
   [junit4]   2> 4002388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoreContainer New CoreContainer 778973569
   [junit4]   2> 4002388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/]
   [junit4]   2> 4002388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/lib
   [junit4]   2> 4002389 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/lib).
   [junit4]   2> 4002399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4002401 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4002401 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4002402 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4002402 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55147/solr
   [junit4]   2> 4002402 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4002402 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4002403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4002405 INFO  (zkCallback-2288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12ce0e17 name:ZooKeeperConnection Watcher:127.0.0.1:55147 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4002405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4002405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4002407 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4002409 INFO  (zkCallback-2289-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79a8a2ae name:ZooKeeperConnection Watcher:127.0.0.1:55147/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4002409 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4002411 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 4002413 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 4002417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 4002420 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 4002422 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 4002426 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 4002428 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 4002430 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 4002432 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 4002434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44227_ckb%2Fp
   [junit4]   2> 4002435 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44227_ckb%2Fp
   [junit4]   2> 4002437 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 4002439 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 4002441 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4002442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000
   [junit4]   2> 4002442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44227_ckb%2Fp
   [junit4]   2> 4002442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 4002444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer Overseer (id=94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) starting
   [junit4]   2> 4002447 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 4002455 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4002456 INFO  (OverseerCollectionConfigSetProcessor-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 4002456 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4002457 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 4002498 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4002499 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4002500 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores
   [junit4]   2> 4002501 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores/collection1/, schema=schema.xml, coreNodeName=, dataDir=data/, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores/collection1, configSetProperties=configsetprops.json, config=solrconfig.xml, name=collection1, loadOnStartup=true, transient=false}
   [junit4]   2> 4002501 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores/collection1/
   [junit4]   2> 4002501 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4002503 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4002503 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4002503 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4002503 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4002506 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:44227/ckb/p",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:44227_ckb%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 4002506 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4002506 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:44227/ckb/p",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:44227_ckb%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4002507 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4002507 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4002509 INFO  (zkCallback-2289-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4003507 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4003507 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 4003508 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4003508 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4003508 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4003509 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores/collection1/'
   [junit4]   2> 4003528 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4003541 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4003562 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 4003612 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4003614 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 4003625 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4003923 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4003931 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 4003933 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 4003943 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4003946 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4003949 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4003950 WARN  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4003950 WARN  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4003950 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4003951 WARN  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4003951 WARN  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4003952 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4003952 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4003953 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 4003953 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:43965/solr_hdfs_home
   [junit4]   2> 4003954 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4003954 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 4003954 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c807a29
   [junit4]   2> 4003954 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4003970 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 4003971 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4003972 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 4003982 WARN  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:43965/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4003982 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4003995 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 4003995 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4003995 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=25, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 4004010 INFO  (IPC Server handler 4 on 43965) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:37771 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5eb5d3af-17a5-4720-8323-eaa0e27a5d03:NORMAL:127.0.0.1:43074|RBW], ReplicaUnderConstruction[[DISK]DS-929254c3-073f-4cad-ab4d-7d714322c01b:NORMAL:127.0.0.1:37771|RBW]]} size 0
   [junit4]   2> 4004012 INFO  (IPC Server handler 5 on 43965) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43074 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5eb5d3af-17a5-4720-8323-eaa0e27a5d03:NORMAL:127.0.0.1:43074|RBW], ReplicaUnderConstruction[[DISK]DS-929254c3-073f-4cad-ab4d-7d714322c01b:NORMAL:127.0.0.1:37771|RBW]]} size 0
   [junit4]   2> 4004017 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b744ce5f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3a96f57f; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4004017 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4004018 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:43965/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 4004024 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4004024 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4004024 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4004024 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4004025 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4004025 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4004025 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4004025 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4004026 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4004028 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4004030 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4004031 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4004033 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4004037 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/admin/luke,/update/csv,/admin/file,/admin/plugins,/update/json,/config,/admin/ping,/admin/properties,/replication,/admin/logging,/admin/mbeans,/get,/admin/system,/schema,/update,/update/json/docs,standard,/admin/threads
   [junit4]   2> 4004052 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 4004052 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4004053 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 4004066 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 4004067 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 4004068 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=688863864, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 4004074 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b744ce5f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3a96f57f; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4004074 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4004075 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@32d0b3f0[collection1] main
   [junit4]   2> 4004075 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4004076 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4004076 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4004076 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4004077 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 4004077 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4004077 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4004078 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4004078 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 4004078 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4004079 INFO  (searcherExecutor-7082-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@32d0b3f0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4004081 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 4004081 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 4004081 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 4004081 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1512557446261899264
   [junit4]   2> 4004081 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1512557446261899264
   [junit4]   2> 4004082 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 4004082 INFO  (coreLoadExecutor-7081-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 4004083 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44227/ckb/p collection:control_collection shard:shard1
   [junit4]   2> 4004084 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 4004087 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94534840245288964-core_node1-n_0000000000
   [junit4]   2> 4004088 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 4004089 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4004089 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4004090 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44227/ckb/p/collection1/
   [junit4]   2> 4004090 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 4004090 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4004090 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44227/ckb/p/collection1/ has no replicas
   [junit4]   2> 4004090 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44227/ckb/p/collection1/ shard1
   [junit4]   2> 4004091 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 4004091 INFO  (zkCallback-2289-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4004093 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 4004096 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44227/ckb/p",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 4004200 INFO  (zkCallback-2289-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4004247 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44227/ckb/p/collection1/ and leader is http://127.0.0.1:44227/ckb/p/collection1/
   [junit4]   2> 4004247 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44227/ckb/p
   [junit4]   2> 4004248 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4004248 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 4004248 INFO  (coreZkRegister-7075-thread-1-processing-c:control_collection n:127.0.0.1:44227_ckb%2Fp x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:44227_ckb%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4004250 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"http://127.0.0.1:44227/ckb/p",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:44227_ckb%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 4004251 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"http://127.0.0.1:44227/ckb/p",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:44227_ckb%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4004354 INFO  (zkCallback-2289-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4004506 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4004506 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4004508 INFO  (zkCallback-2291-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1b8256ba name:ZooKeeperConnection Watcher:127.0.0.1:55147/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4004508 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4004509 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4004509 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4004512 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 4004813 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores/collection1
   [junit4]   2> 4004814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001
   [junit4]   2> 4004815 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4004817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@58463eb{/ckb/p,null,AVAILABLE}
   [junit4]   2> 4004818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@67711699{HTTP/1.1}{127.0.0.1:42638}
   [junit4]   2> 4004818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.e.j.s.Server Started @4007992ms
   [junit4]   2> 4004818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=42638, solr.data.dir=hdfs://localhost:43965/hdfs__localhost_43965__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001_tempDir-002_jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores, hostContext=/ckb/p, solrconfig=solrconfig.xml}
   [junit4]   2> 4004818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 4004818 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/'
   [junit4]   2> 4004834 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4004836 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4004837 INFO  (zkCallback-2292-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b85db39 name:ZooKeeperConnection Watcher:127.0.0.1:55147/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4004837 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4004838 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4004839 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4004839 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/solr.xml
   [junit4]   2> 4004853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores
   [junit4]   2> 4004854 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoreContainer New CoreContainer 77074140
   [junit4]   2> 4004854 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/]
   [junit4]   2> 4004854 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/lib
   [junit4]   2> 4004854 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/lib).
   [junit4]   2> 4004862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4004863 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4004864 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4004864 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4004864 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55147/solr
   [junit4]   2> 4004864 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4004865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4004865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4004867 INFO  (zkCallback-2294-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27f47967 name:ZooKeeperConnection Watcher:127.0.0.1:55147 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4004867 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4004867 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4004869 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4004870 INFO  (zkCallback-2295-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6bda2447 name:ZooKeeperConnection Watcher:127.0.0.1:55147/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4004871 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4004876 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4005881 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42638_ckb%2Fp
   [junit4]   2> 4005882 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:42638_ckb%2Fp
   [junit4]   2> 4005883 INFO  (zkCallback-2289-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4005883 INFO  (zkCallback-2291-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4005883 INFO  (zkCallback-2295-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4005885 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4005886 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94534840245288968-127.0.0.1:42638_ckb%2Fp-n_0000000001
   [junit4]   2> 4005887 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000 to know if I could be the leader
   [junit4]   2> 4005904 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4005905 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4005906 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores
   [junit4]   2> 4005907 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores/collection1/, loadOnStartup=true, collection=collection1, schema=schema.xml, transient=false, config=solrconfig.xml, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores/collection1, dataDir=data/, coreNodeName=, shard=, name=collection1}
   [junit4]   2> 4005907 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores/collection1/
   [junit4]   2> 4005907 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4005908 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4005908 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4005909 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4005908 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[428DD5848704966F]) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4005910 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42638/ckb/p",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:42638_ckb%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 4005910 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4005911 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42638/ckb/p",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:42638_ckb%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4005911 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 4005911 INFO  (OverseerStateUpdate-94534840245288964-127.0.0.1:44227_ckb%2Fp-n_0000000000) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 4005912 INFO  (zkCallback-2291-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4005912 INFO  (zkCallback-2289-thread-1-processing-n:127.0.0.1:44227_ckb%2Fp) [n:127.0.0.1:44227_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4005913 INFO  (zkCallback-2295-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4006912 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4006912 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 4006913 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4006913 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 4006914 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4006914 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_428DD5848704966F-001/shard-1-001/cores/collection1/'
   [junit4]   2> 4006933 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4006941 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4006955 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 4006986 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4006987 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42638_ckb%2Fp c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 4006995 INFO  (coreLoadExecutor-7092-thread-1-processing-n:127.0.0.1:42638_ckb%2Fp) [n:127.0.0.1:42

[...truncated too long message...]

e/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150917_054621_217.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150917_054621_217.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 224 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1865246453 bytes
Compression is 0.0%
Took 5 min 42 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 957 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/957/

2 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=77233, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=77233, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37034/_l/j: Could not find collection : awholynewstresscollection_collection3_1
	at __randomizedtesting.SeedInfo.seed([33D546A4102F2C79]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=108998, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=108998, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:46708/lv_/kb: collection already exists: awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([33D546A4102F2C79]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 11377 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/init-core-data-001
   [junit4]   2> 1650624 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[33D546A4102F2C79]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1650625 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[33D546A4102F2C79]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /lv_/kb
   [junit4]   2> 1650629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1650630 INFO  (Thread-104049) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1650630 INFO  (Thread-104049) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1650730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.ZkTestServer start zk server on port:52782
   [junit4]   2> 1650730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1650732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1650736 INFO  (zkCallback-926-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20a4e7f4 name:ZooKeeperConnection Watcher:127.0.0.1:52782 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1650736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1650736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1650736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1650742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1650743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1650743 INFO  (zkCallback-927-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2740684a name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1650744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1650744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1650744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1650746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1650748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1650749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1650751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1650751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1650757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1650757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1650762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1650762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1650769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1650769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1650771 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1650771 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1650774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1650774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1650777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1650777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1650781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1650781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1650784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1650784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1650790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1650790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1650793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1650793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1650797 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1650798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1650799 INFO  (zkCallback-928-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@260e1d51 name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1650799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1650800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1651074 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1
   [junit4]   2> 1651076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1651097 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@604fbb0b{/lv_/kb,null,AVAILABLE}
   [junit4]   2> 1651509 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1177fbf4{SSL-http/1.1}{127.0.0.1:46708}
   [junit4]   2> 1651510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.Server Started @1654447ms
   [junit4]   2> 1651510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores, hostPort=46708, hostContext=/lv_/kb, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/tempDir-001/control/data}
   [junit4]   2> 1651510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1651510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/'
   [junit4]   2> 1651535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1651537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1651539 INFO  (zkCallback-929-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@434bc69 name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1651540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1651540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1651544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1651544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/solr.xml
   [junit4]   2> 1651567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores
   [junit4]   2> 1651567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoreContainer New CoreContainer 414609685
   [junit4]   2> 1651567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/]
   [junit4]   2> 1651567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/lib
   [junit4]   2> 1651567 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/lib).
   [junit4]   2> 1651580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1651583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1651585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1651585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1651585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52782/solr
   [junit4]   2> 1651585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1651586 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1651587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1651590 INFO  (zkCallback-931-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4f9d54b6 name:ZooKeeperConnection Watcher:127.0.0.1:52782 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1651592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1651593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1651595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1651597 INFO  (zkCallback-932-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77e51ad4 name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1651597 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1651599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1651603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1651607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1651610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1651628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1651635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1651639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1651641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1651644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1651645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46708_lv_%2Fkb
   [junit4]   2> 1651646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46708_lv_%2Fkb
   [junit4]   2> 1651648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1651650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1651651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1651652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000
   [junit4]   2> 1651653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46708_lv_%2Fkb
   [junit4]   2> 1651653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1651658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer Overseer (id=94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) starting
   [junit4]   2> 1651661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1651675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1651684 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1651685 INFO  (OverseerCollectionConfigSetProcessor-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1651687 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1651718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1651718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1651719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores
   [junit4]   2> 1651720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, collection=control_collection, config=solrconfig.xml, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1, schema=schema.xml, shard=, transient=false, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/, dataDir=data/}
   [junit4]   2> 1651720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/
   [junit4]   2> 1651720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1651722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1651722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1651722 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1651723 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1651726 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:46708/lv_/kb",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:46708_lv_%2Fkb",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1651726 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1651727 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:46708/lv_/kb",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:46708_lv_%2Fkb",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1651727 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1651728 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1651730 INFO  (zkCallback-932-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1652727 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1652727 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1652728 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1652728 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1652729 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1652729 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/'
   [junit4]   2> 1652747 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1652755 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1652775 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1652819 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1652821 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1652832 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1653143 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1653153 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1653155 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1653167 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1653171 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1653175 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1653176 WARN  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1653176 WARN  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1653176 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1653177 WARN  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1653177 WARN  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1653178 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1653178 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1653178 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1653178 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1653179 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a130ef5
   [junit4]   2> 1653179 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/data
   [junit4]   2> 1653179 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1653180 WARN  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1653180 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/data/index
   [junit4]   2> 1653180 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=32, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1368388848134905]
   [junit4]   2> 1653181 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@488e0c03 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65b6be2b),segFN=segments_1,generation=1}
   [junit4]   2> 1653181 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1653185 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/data/
   [junit4]   2> 1653185 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1653187 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1653187 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1653187 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1653188 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1653188 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1653188 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1653188 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1653188 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1653189 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1653191 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1653193 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1653194 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1653195 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1653201 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/file,/admin/ping,/get,/admin/plugins,standard,/update/json,/admin/system,/update/json/docs,/schema,/admin/logging,/config,/admin/mbeans,/replication,/admin/properties,/update/csv,/update,/admin/segments,/admin/threads,/admin/luke
   [junit4]   2> 1653202 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1653208 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1653208 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1653209 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1653209 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1653210 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=50.1044921875, floorSegmentMB=1.4833984375, forceMergeDeletesPctAllowed=20.66700176464544, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1653211 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@488e0c03 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65b6be2b),segFN=segments_1,generation=1}
   [junit4]   2> 1653211 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1653211 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5415d5a9[collection1] main
   [junit4]   2> 1653211 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1653212 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1653212 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1653212 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1653212 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1653213 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1653213 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1653213 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1653213 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1653214 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1653214 INFO  (searcherExecutor-2823-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5415d5a9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1653215 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1653215 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1653215 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1653215 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1512425877597585408
   [junit4]   2> 1653217 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1512425877597585408
   [junit4]   2> 1653217 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1653218 INFO  (coreLoadExecutor-2822-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1653220 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:46708/lv_/kb collection:control_collection shard:shard1
   [junit4]   2> 1653221 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1653227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1653228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1653230 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94526617180504069-core_node1-n_0000000000
   [junit4]   2> 1653230 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1653232 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1653232 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1653232 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:46708/lv_/kb/collection1/
   [junit4]   2> 1653233 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1653233 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1653233 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:46708/lv_/kb/collection1/ has no replicas
   [junit4]   2> 1653233 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:46708/lv_/kb/collection1/ shard1
   [junit4]   2> 1653235 INFO  (zkCallback-932-thread-1-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1653236 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1653238 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1653243 INFO  (zkCallback-934-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1570670a name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1653243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1653243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1653243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1653243 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:46708/lv_/kb",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1653246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1653247 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1653247 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1653248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1653248 INFO  (zkCallback-935-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@713022d3 name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1653249 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1653249 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1653256 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 1653256 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1653257 INFO  (zkCallback-934-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1653258 INFO  (zkCallback-932-thread-2-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1653291 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:46708/lv_/kb/collection1/ and leader is https://127.0.0.1:46708/lv_/kb/collection1/
   [junit4]   2> 1653291 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:46708/lv_/kb
   [junit4]   2> 1653291 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1653291 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1653291 INFO  (coreZkRegister-2816-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1653294 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"https://127.0.0.1:46708/lv_/kb",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:46708_lv_%2Fkb",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 1653296 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"https://127.0.0.1:46708/lv_/kb",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:46708_lv_%2Fkb",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1653297 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1653402 INFO  (zkCallback-932-thread-2-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1653402 INFO  (zkCallback-934-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1653650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores/collection1
   [junit4]   2> 1653651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001
   [junit4]   2> 1653652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1653659 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49c98fc8{/lv_/kb,null,AVAILABLE}
   [junit4]   2> 1653663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.ServerConnector Started ServerConnector@280ea9ff{SSL-http/1.1}{127.0.0.1:58579}
   [junit4]   2> 1653663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.e.j.s.Server Started @1656600ms
   [junit4]   2> 1653663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/lv_/kb, hostPort=58579, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/tempDir-001/jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 1653664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1653665 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/'
   [junit4]   2> 1653704 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1653705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1653711 INFO  (zkCallback-936-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@229f0730 name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1653711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1653711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1653713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1653713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/solr.xml
   [junit4]   2> 1653762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores
   [junit4]   2> 1653762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoreContainer New CoreContainer 917384387
   [junit4]   2> 1653762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/]
   [junit4]   2> 1653763 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/lib
   [junit4]   2> 1653763 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/lib).
   [junit4]   2> 1653772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1653775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1653776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1653776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1653777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52782/solr
   [junit4]   2> 1653777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1653777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1653784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1653800 INFO  (zkCallback-938-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7db04792 name:ZooKeeperConnection Watcher:127.0.0.1:52782 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1653801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1653801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1653829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1653836 INFO  (zkCallback-939-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40c97bc6 name:ZooKeeperConnection Watcher:127.0.0.1:52782/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1653836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1653841 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1654846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58579_lv_%2Fkb
   [junit4]   2> 1654847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58579_lv_%2Fkb
   [junit4]   2> 1654848 INFO  (zkCallback-932-thread-2-processing-n:127.0.0.1:46708_lv_%2Fkb) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1654848 INFO  (zkCallback-939-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1654849 INFO  (zkCallback-934-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1654851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1654852 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94526617180504074-127.0.0.1:58579_lv_%2Fkb-n_0000000001
   [junit4]   2> 1654853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000 to know if I could be the leader
   [junit4]   2> 1654875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1654875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1654876 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores
   [junit4]   2> 1654877 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores/collection1/, configSetProperties=configsetprops.json, config=solrconfig.xml, loadOnStartup=true, dataDir=data/, collection=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores/collection1, shard=, coreNodeName=, transient=false}
   [junit4]   2> 1654877 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores/collection1/
   [junit4]   2> 1654878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1654879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1654879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[33D546A4102F2C79]) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1654880 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1654880 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1654882 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:58579/lv_/kb",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:58579_lv_%2Fkb",
   [junit4]   2>   "state":"down"} current state version: 4
   [junit4]   2> 1654883 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:58579/lv_/kb",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:58579_lv_%2Fkb",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1654883 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1654883 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1654884 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1654884 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1654885 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1654885 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1654986 INFO  (OverseerStateUpdate-94526617180504069-127.0.0.1:46708_lv_%2Fkb-n_0000000000) [n:127.0.0.1:46708_lv_%2Fkb    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1654986 INFO  (zkCallback-939-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1654987 INFO  (zkCallback-939-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1655886 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1655886 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1655886 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1655886 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1655887 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1655887 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_33D546A4102F2C79-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1655905 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1655912 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1655928 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1655957 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1655958 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1655965 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1656261 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1656278 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1656281 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1656294 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1656298 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1656302 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1656303 WARN  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1656303 WARN  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1656304 INFO  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb) [n:127.0.0.1:58579_lv_%2Fkb c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1656305 WARN  (coreLoadExecutor-2833-thread-1-processing-n:127.0.0.1:58579_lv_%2Fkb

[...truncated too long message...]

Util.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4362301 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[33D546A4102F2C79]-worker) [n:127.0.0.1:50601__l%2Fj c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 15, 2015 8:47:04 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_l=Lucene50(blocksize=128), other_tl1=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=Asserting), text=Lucene50(blocksize=128), _version_=PostingsFormat(name=Asserting), a_t=Lucene50(blocksize=128), id=FST50, timestamp=Lucene50(blocksize=128), range_facet_l_dv=FST50, multiDefault=Lucene50(blocksize=128), range_facet_i_dv=Lucene50(blocksize=128), a_i1=Lucene50(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=sr_CS, timezone=Europe/Madrid
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=64891848,total=534249472
   [junit4]   2> NOTE: All tests run in this JVM: [TestPerFieldSimilarity, TestStressRecovery, TestJmxIntegration, XmlUpdateRequestHandlerTest, ZkNodePropsTest, TestConfigReload, ConcurrentDeleteAndCreateCollectionTest, TestTrackingShardHandlerFactory, DistributedExpandComponentTest, NoCacheHeaderTest, TestFuzzyAnalyzedSuggestions, TestCloudPivotFacet, SampleTest, RequestHandlersTest, CoreAdminCreateDiscoverTest, DirectSolrConnectionTest, TestDistribDocBasedVersion, FieldAnalysisRequestHandlerTest, QueryParsingTest, TestIndexingPerformance, HdfsNNFailoverTest, MultiThreadedOCPTest, DeleteInactiveReplicaTest, TestInitQParser, SolrXmlInZkTest, TestSerializedLuceneMatchVersion, TestMaxScoreQueryParser, LukeRequestHandlerTest, UpdateParamsTest, TestClusterStateMutator, TestManagedSchemaDynamicFieldResource, TestSweetSpotSimilarityFactory, HighlighterConfigTest, TestReversedWildcardFilterFactory, ExplicitHLLTest, TestLRUStatsCache, TestSchemaSimilarityResource, TestFreeTextSuggestions, CoreAdminHandlerTest, TestRestoreCore, BadComponentTest, SolrCoreCheckLockOnStartupTest, TestObjectReleaseTracker, ZkStateWriterTest, FieldMutatingUpdateProcessorTest, SharedFSAutoReplicaFailoverUtilsTest, DirectUpdateHandlerOptimizeTest, TestHdfsUpdateLog, BinaryUpdateRequestHandlerTest, AliasIntegrationTest, UUIDFieldTest, TestSchemaNameResource, SystemInfoHandlerTest, TestSmileRequest, DistributedFacetPivotLargeTest, TestFastWriter, BigEndianAscendingWordDeserializerTest, PrimitiveFieldTypeTest, TestSystemIdResolver, TestTrieFacet, TestCrossCoreJoin, TestRawResponseWriter, TestRandomRequestDistribution, TestComplexPhraseQParserPlugin, DeleteShardTest, FullSolrCloudDistribCmdsTest, BasicDistributedZk2Test, TestExpandComponent, HdfsChaosMonkeySafeLeaderTest, SuggesterWFSTTest, CollectionTooManyReplicasTest, TestGroupingSearch, DistributedQueryComponentCustomSortTest, UnloadDistributedZkTest, RecoveryAfterSoftCommitTest, ClusterStateTest, TestSolr4Spatial2, TestDefaultSimilarityFactory, TestReload, TestCryptoKeys, PluginInfoTest, DistributedDebugComponentTest, CursorPagingTest, RemoteQueryErrorTest, DistributedQueryComponentOptimizationTest, SharedFSAutoReplicaFailoverTest, TestFastLRUCache, HLLSerializationTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, TestDistributedSearch, TestRecovery, TestHashPartitioner, ZkControllerTest, TestStressReorder, HardAutoCommitTest, TestRangeQuery, TestSolr4Spatial, SolrCmdDistributorTest, PeerSyncTest, BadIndexSchemaTest, TestSort, TestFiltering, TestFunctionQuery, TestBadConfig, SoftAutoCommitTest, CurrencyFieldXmlFileTest, TestExtendedDismaxParser, SpellCheckCollatorTest, TestFoldingMultitermQuery, SuggesterTest, PolyFieldTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestSolrDeletionPolicy1, CacheHeaderTest, TestSurroundQueryParser, TestQueryUtils, TestWriterPerf, TermsComponentTest, TestArbitraryIndexDir, SolrPluginUtilsTest, TestCollationField, TestCSVResponseWriter, TestAnalyzedSuggestions, TestPHPSerializedResponseWriter, SearchHandlerTest, TestQuerySenderListener, CopyFieldTest, TestSolrDeletionPolicy2, TestDocSet, TestSearchPerf, MinimalSchemaTest, TestConfig, NotRequiredUniqueKeyTest, TestCharFilters, SynonymTokenizerTest, TestDFRSimilarityFactory, TestBM25SimilarityFactory, ResourceLoaderTest, OpenExchangeRatesOrgProviderTest, ChaosMonkeyNothingIsSafeTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestHighlightDedupGrouping, TestEmbeddedSolrServerConstructors, AsyncMigrateRouteKeyTest, CollectionsAPIAsyncDistributedZkTest, DeleteReplicaTest, DistribCursorPagingTest, DistributedVersionInfoTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, OverseerStatusTest, OverseerTaskQueueTest, SaslZkACLProviderTest, SimpleCollectionCreateDeleteTest, TestCloudInspectUtil, TestCollectionAPI, TestConfigSetsAPI, TestLeaderInitiatedRecoveryThread, TestMiniSolrCloudClusterBase, TestRebalanceLeaders, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [427/538] on J2 in 414.52s, 1 test, 1 error <<< FAILURES!

[...truncated 346 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 538 suites, 2127 tests, 2 errors, 29 ignored (17 assumptions)

Total time: 198 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1183108875 bytes
Compression is 0.0%
Took 3 min 12 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 956 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/956/

2 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2758, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2758, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38206: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([25F1FD9DB73A8F95]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=9518, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9518, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:52357/_gha/f: collection already exists: awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([25F1FD9DB73A8F95]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 10342 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/init-core-data-001
   [junit4]   2> 130268 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 130268 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 131013 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 132289 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 132517 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 132527 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 132618 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 132672 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_35010_hdfs____.e77hd5/webapp
   [junit4]   2> 132888 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 133782 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35010
   [junit4]   2> 134920 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 134928 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 134946 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57824_datanode____.2bgght/webapp
   [junit4]   2> 135092 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 135629 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57824
   [junit4]   2> 135760 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 135763 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 135822 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33316_datanode____2qzqcf/webapp
   [junit4]   2> 135955 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 136501 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[25F1FD9DB73A8F95]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33316
   [junit4]   2> 136938 INFO  (IPC Server handler 4 on 41386) [    ] BlockStateChange BLOCK* processReport: from storage DS-ab0efcd7-cfca-472c-b40a-687b698e6b89 node DatanodeRegistration(127.0.0.1, datanodeUuid=5fe1b6e3-d736-4012-9aff-181bb5de60af, infoPort=57824, ipcPort=51839, storageInfo=lv=-56;cid=testClusterID;nsid=2041829524;c=0), blocks: 0, hasStaleStorages: true, processing time: 4 msecs
   [junit4]   2> 136938 INFO  (IPC Server handler 5 on 41386) [    ] BlockStateChange BLOCK* processReport: from storage DS-98eda4f7-56d6-41b7-9728-cda316360311 node DatanodeRegistration(127.0.0.1, datanodeUuid=5a0d42b9-ae0c-4c14-b84a-5a0d844dafd0, infoPort=33316, ipcPort=52626, storageInfo=lv=-56;cid=testClusterID;nsid=2041829524;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 136950 INFO  (IPC Server handler 5 on 41386) [    ] BlockStateChange BLOCK* processReport: from storage DS-54c9f456-6171-4ae2-a512-81f0d3bf2708 node DatanodeRegistration(127.0.0.1, datanodeUuid=5a0d42b9-ae0c-4c14-b84a-5a0d844dafd0, infoPort=33316, ipcPort=52626, storageInfo=lv=-56;cid=testClusterID;nsid=2041829524;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 136955 INFO  (IPC Server handler 4 on 41386) [    ] BlockStateChange BLOCK* processReport: from storage DS-f6994330-1d7f-47f2-bed3-3023d01e4736 node DatanodeRegistration(127.0.0.1, datanodeUuid=5fe1b6e3-d736-4012-9aff-181bb5de60af, infoPort=57824, ipcPort=51839, storageInfo=lv=-56;cid=testClusterID;nsid=2041829524;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 137555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 137563 INFO  (Thread-336) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 137566 INFO  (Thread-336) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 137660 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.ZkTestServer start zk server on port:54862
   [junit4]   2> 137660 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 137676 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 137688 INFO  (zkCallback-67-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5cdf3c40 name:ZooKeeperConnection Watcher:127.0.0.1:54862 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 137688 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 137689 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 137689 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 137696 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 137699 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 137700 INFO  (zkCallback-68-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@280ab145 name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 137708 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 137709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 137710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 137712 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 137715 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 137716 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 137718 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 137719 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 137722 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 137724 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 137727 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 137727 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 137729 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 137732 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 137734 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 137744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 137747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 137747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 137758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 137759 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 137761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 137762 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 137770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 137770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 137773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 137773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 137776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 137776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 137792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 137794 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 137797 INFO  (zkCallback-69-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24d60928 name:ZooKeeperConnection Watcher:127.0.0.1:54862 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 137797 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 137797 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 137798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 137800 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 137802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 137809 INFO  (zkCallback-70-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@23cdac1f name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 137810 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 137810 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 137811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 137811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 137815 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 137815 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 137817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 137817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 137819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 137819 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 137821 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 137821 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 137823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 137824 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 137825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 137826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 137827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 137828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 137830 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 137830 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 137832 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 137833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 137834 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 137835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 137837 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 137837 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 138282 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores/collection1
   [junit4]   2> 138284 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 138286 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@376f247c{/,null,AVAILABLE}
   [junit4]   2> 138287 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.ServerConnector Started ServerConnector@24d81ca3{HTTP/1.1}{127.0.0.1:57420}
   [junit4]   2> 138287 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.Server Started @141473ms
   [junit4]   2> 138287 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores, solr.data.dir=hdfs://localhost:41386/hdfs__localhost_41386__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001_tempDir-002_control_data, hostPort=57420, hostContext=/}
   [junit4]   2> 138288 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 138288 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/'
   [junit4]   2> 138308 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 138308 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 138312 INFO  (zkCallback-71-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8df2c5e name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 138312 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 138313 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 138317 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 138318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/solr.xml
   [junit4]   2> 138335 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores
   [junit4]   2> 138335 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoreContainer New CoreContainer 1878006249
   [junit4]   2> 138335 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/]
   [junit4]   2> 138336 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/lib
   [junit4]   2> 138336 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/lib).
   [junit4]   2> 138346 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 138348 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 138349 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 138349 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 138349 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54862/solr
   [junit4]   2> 138349 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 138349 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 138350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 138352 INFO  (zkCallback-73-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@178e1568 name:ZooKeeperConnection Watcher:127.0.0.1:54862 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 138353 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 138354 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 138365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 138372 INFO  (zkCallback-74-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e972a97 name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 138372 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 138374 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 138377 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 138380 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 138383 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 138385 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 138387 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 138389 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 138390 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 138392 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 138393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57420_
   [junit4]   2> 138394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57420_
   [junit4]   2> 138396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 138397 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 138399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 138399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94517073506205702-127.0.0.1:57420_-n_0000000000
   [junit4]   2> 138400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57420_
   [junit4]   2> 138400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 138402 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer Overseer (id=94517073506205702-127.0.0.1:57420_-n_0000000000) starting
   [junit4]   2> 138404 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 138411 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 138413 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 138413 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 138416 INFO  (OverseerCollectionConfigSetProcessor-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 138425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 138444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 138444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 138446 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores
   [junit4]   2> 138447 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores/collection1, shard=, coreNodeName=, name=collection1, transient=false, dataDir=data/, collection=control_collection, config=solrconfig.xml, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores/collection1/, schema=schema.xml}
   [junit4]   2> 138447 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores/collection1/
   [junit4]   2> 138447 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 138448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 138449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:57420_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 138450 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 138451 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 138453 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:57420_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57420",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 138454 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:57420_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57420",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 138454 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 138455 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 138456 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 138459 INFO  (zkCallback-74-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 138459 INFO  (zkCallback-74-thread-2-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 139456 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 139457 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 139457 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 139458 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 139458 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 139459 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores/collection1/'
   [junit4]   2> 139477 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 139492 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 139506 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 139538 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 139541 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 139555 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 140258 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 140268 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 140295 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 140375 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 140379 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 140382 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 140383 WARN  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 140383 WARN  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 140384 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 140384 WARN  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 140385 WARN  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 140386 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 140386 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 140386 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 140398 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:41386/solr_hdfs_home
   [junit4]   2> 140399 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 140399 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 140399 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@38aa594b
   [junit4]   2> 140416 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 140693 WARN  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 145739 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 145739 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 145745 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 145779 WARN  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:41386/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 145779 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 145799 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 145800 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 145800 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@986112a
   [junit4]   2> 146152 INFO  (IPC Server handler 7 on 41386) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51052 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-54c9f456-6171-4ae2-a512-81f0d3bf2708:NORMAL:127.0.0.1:52828|RBW], ReplicaUnderConstruction[[DISK]DS-f6994330-1d7f-47f2-bed3-3023d01e4736:NORMAL:127.0.0.1:51052|RBW]]} size 71
   [junit4]   2> 146154 INFO  (IPC Server handler 6 on 41386) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52828 is added to blk_1073741825_1001 size 71
   [junit4]   2> 146559 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6f81bb8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70f9aae; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 146559 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 146563 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:41386/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 146578 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 146578 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 146579 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 146579 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 146579 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 146579 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 146580 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 146580 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 146580 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 146584 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 146586 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 146587 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 146589 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 146596 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/admin/properties,/admin/threads,/admin/luke,/admin/file,/config,standard,/get,/admin/mbeans,/update/json/docs,/update/json,/update/csv,/admin/system,/replication,/schema,/admin/ping,/update,/admin/plugins,/admin/segments
   [junit4]   2> 146600 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 146603 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 146604 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 146641 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 146641 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 146645 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=21, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4852792598818342]
   [junit4]   2> 146727 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6f81bb8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70f9aae; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 146727 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 146728 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@23da9489[collection1] main
   [junit4]   2> 146728 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 146730 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 146730 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 146732 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 146732 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 146732 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 146734 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 146735 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 146735 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 146735 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 146737 INFO  (searcherExecutor-275-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@23da9489[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 146746 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 146746 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 146746 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 146746 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1512273185723121664
   [junit4]   2> 146746 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1512273185723121664
   [junit4]   2> 146747 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 146747 INFO  (coreLoadExecutor-274-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 146748 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57420 collection:control_collection shard:shard1
   [junit4]   2> 146749 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 146757 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94517073506205702-core_node1-n_0000000000
   [junit4]   2> 146760 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 146767 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 146768 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 146768 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57420/collection1/
   [junit4]   2> 146768 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 146768 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57420/collection1/ has no replicas
   [junit4]   2> 146768 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57420/collection1/ shard1
   [junit4]   2> 146767 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 146773 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 146774 INFO  (zkCallback-74-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 146774 INFO  (zkCallback-74-thread-2-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 146779 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 146782 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57420",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 146885 INFO  (zkCallback-74-thread-2-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 146886 INFO  (zkCallback-74-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 146933 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57420/collection1/ and leader is http://127.0.0.1:57420/collection1/
   [junit4]   2> 146933 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57420
   [junit4]   2> 146933 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 146933 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 146934 INFO  (coreZkRegister-268-thread-1-processing-n:127.0.0.1:57420_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:57420_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 146936 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:57420_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57420",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 146937 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:57420_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57420",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 146955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 146956 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 146957 INFO  (zkCallback-76-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6962a3cc name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 146957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 146958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 146958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 146961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 146962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 146962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 146963 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 146965 INFO  (zkCallback-77-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48c35c50 name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 146965 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 146966 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 146970 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 146970 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 146971 INFO  (zkCallback-76-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 146971 INFO  (zkCallback-74-thread-2-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 146972 INFO  (zkCallback-74-thread-1-processing-n:127.0.0.1:57420_) [n:127.0.0.1:57420_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 147078 INFO  (OverseerStateUpdate-94517073506205702-127.0.0.1:57420_-n_0000000000) [n:127.0.0.1:57420_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 147342 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/cores/collection1
   [junit4]   2> 147344 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001
   [junit4]   2> 147346 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 147357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b7c3088{/,null,AVAILABLE}
   [junit4]   2> 147358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.ServerConnector Started ServerConnector@18360662{HTTP/1.1}{127.0.0.1:38206}
   [junit4]   2> 147358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.e.j.s.Server Started @150544ms
   [junit4]   2> 147358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:41386/hdfs__localhost_41386__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001_tempDir-002_jetty1, hostPort=38206, solrconfig=solrconfig.xml}
   [junit4]   2> 147359 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 147359 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/'
   [junit4]   2> 147392 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 147394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 147398 INFO  (zkCallback-78-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@ac4dc1e name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 147399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 147400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 147403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 147404 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/solr.xml
   [junit4]   2> 147428 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/cores
   [junit4]   2> 147429 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoreContainer New CoreContainer 1059015670
   [junit4]   2> 147429 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/]
   [junit4]   2> 147430 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/lib
   [junit4]   2> 147430 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_25F1FD9DB73A8F95-001/shard-1-001/lib).
   [junit4]   2> 147444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 147446 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 147448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 147448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 147448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54862/solr
   [junit4]   2> 147449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 147449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 147450 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 147453 INFO  (zkCallback-80-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3433a752 name:ZooKeeperConnection Watcher:127.0.0.1:54862 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 147454 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 147455 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 147458 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:38206_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 147460 INFO  (zkCallback-81-thread-1-processing-n:127.0.0.1:38206_) [n:127.0.0.1:38206_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a9356a6 name:ZooKeeperConnection Watcher:127.0.0.1:54862/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 147460 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:38206_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 147472 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:38206_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 148483 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:38206_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38206_
   [junit4]   2> 148484 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[25F1FD9DB73A8F95]) [n:127.0.0.1:38206_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38206_
   [junit4]   2> 148485 INFO  (zkCallback-76-thread-1) [    ] o.

[...truncated too long message...]

ave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150914_033226_569.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150914_033226_569.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 493 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
ERROR: Failed to archive {README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J0-20150914_033226_564.events=solr/build/solr-core/test/temp/junit4-J0-20150914_033226_564.events, solr/build/solr-core/test/temp/junit4-J1-20150914_033226_564.events=solr/build/solr-core/test/temp/junit4-J1-20150914_033226_564.events, solr/build/solr-core/test/temp/junit4-J2-20150914_033226_569.events=solr/build/solr-core/test/temp/junit4-J2-20150914_033226_569.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
	at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
	at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
	at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:145)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException: unexpected EOF with 1536 bytes unread
	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:349)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 13 more
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 955 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/955/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:46937: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46937: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1442111477793,generation=2,filelist=[_7y.cfe, _7y.cfs, _7y.si, _7z.cfe, _7z.cfs, _7z.si, _80.cfe, _80.cfs, _80.si, _81.cfe, _81.cfs, _81.si, _82.cfe, _82.cfs, _82.si, _83.cfe, _83.cfs, _83.si, _84.cfe, _84.cfs, _84.si, _85.cfe, _85.cfs, _85.si, _86.cfe, _86.cfs, _86.si, _87.cfe, _87.cfs, _87.si, _88.cfe, _88.cfs, _88.si, _89.cfe, _89.cfs, _89.si, _8a.cfe, _8a.cfs, _8a.si, _8b.cfe, _8b.cfs, _8b.si, _8d.cfe, _8d.cfs, _8d.si, segments_2]}]> but was:<[{indexVersion=1442111477793,generation=2,filelist=[_7y.cfe, _7y.cfs, _7y.si, _7z.cfe, _7z.cfs, _7z.si, _80.cfe, _80.cfs, _80.si, _81.cfe, _81.cfs, _81.si, _82.cfe, _82.cfs, _82.si, _83.cfe, _83.cfs, _83.si, _84.cfe, _84.cfs, _84.si, _85.cfe, _85.cfs, _85.si, _86.cfe, _86.cfs, _86.si, _87.cfe, _87.cfs, _87.si, _88.cfe, _88.cfs, _88.si, _89.cfe, _89.cfs, _89.si, _8a.cfe, _8a.cfs, _8a.si, _8b.cfe, _8b.cfs, _8b.si, _8d.cfe, _8d.cfs, _8d.si, segments_2]}, {indexVersion=1442111477793,generation=3,filelist=[_8b.cfe, _8b.cfs, _8b.si, _8c.cfe, _8c.cfs, _8c.si, _8d.cfe, _8d.cfs, _8d.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1442111477793,generation=2,filelist=[_7y.cfe, _7y.cfs, _7y.si, _7z.cfe, _7z.cfs, _7z.si, _80.cfe, _80.cfs, _80.si, _81.cfe, _81.cfs, _81.si, _82.cfe, _82.cfs, _82.si, _83.cfe, _83.cfs, _83.si, _84.cfe, _84.cfs, _84.si, _85.cfe, _85.cfs, _85.si, _86.cfe, _86.cfs, _86.si, _87.cfe, _87.cfs, _87.si, _88.cfe, _88.cfs, _88.si, _89.cfe, _89.cfs, _89.si, _8a.cfe, _8a.cfs, _8a.si, _8b.cfe, _8b.cfs, _8b.si, _8d.cfe, _8d.cfs, _8d.si, segments_2]}]> but was:<[{indexVersion=1442111477793,generation=2,filelist=[_7y.cfe, _7y.cfs, _7y.si, _7z.cfe, _7z.cfs, _7z.si, _80.cfe, _80.cfs, _80.si, _81.cfe, _81.cfs, _81.si, _82.cfe, _82.cfs, _82.si, _83.cfe, _83.cfs, _83.si, _84.cfe, _84.cfs, _84.si, _85.cfe, _85.cfs, _85.si, _86.cfe, _86.cfs, _86.si, _87.cfe, _87.cfs, _87.si, _88.cfe, _88.cfs, _88.si, _89.cfe, _89.cfs, _89.si, _8a.cfe, _8a.cfs, _8a.si, _8b.cfe, _8b.cfs, _8b.si, _8d.cfe, _8d.cfs, _8d.si, segments_2]}, {indexVersion=1442111477793,generation=3,filelist=[_8b.cfe, _8b.cfs, _8b.si, _8c.cfe, _8c.cfs, _8c.si, _8d.cfe, _8d.cfs, _8d.si, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([331E276C7B3545BA:16C93C5C0B7D4BB9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10403 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/init-core-data-001
   [junit4]   2> 408516 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[331E276C7B3545BA]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 408516 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[331E276C7B3545BA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 408521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 408522 INFO  (Thread-3404) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 408522 INFO  (Thread-3404) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 408622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.ZkTestServer start zk server on port:35378
   [junit4]   2> 408622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 408638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 408728 INFO  (zkCallback-1579-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@629cc43b name:ZooKeeperConnection Watcher:127.0.0.1:35378 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 408729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 408729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 408729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 408732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 408733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 408740 INFO  (zkCallback-1580-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@501e41bc name:ZooKeeperConnection Watcher:127.0.0.1:35378/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 408740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 408740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 408740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 408743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 408744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 408745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 408747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 408747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 408749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 408749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 408751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 408751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 408753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 408753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 408754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 408755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 408756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 408756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 408758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 408758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 408759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 408759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 408761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 408761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 408763 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 408763 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 408764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 408764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 409161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1
   [junit4]   2> 409163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 409217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f1542b8{/,null,AVAILABLE}
   [junit4]   2> 409219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@26734ae3{HTTP/1.1}{127.0.0.1:51235}
   [junit4]   2> 409219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.Server Started @412996ms
   [junit4]   2> 409219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=51235, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/tempDir-001/control/data}
   [junit4]   2> 409220 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 409220 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/'
   [junit4]   2> 409236 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 409238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 409246 INFO  (zkCallback-1581-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@60d15db9 name:ZooKeeperConnection Watcher:127.0.0.1:35378/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 409246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 409247 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 409248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 409248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/solr.xml
   [junit4]   2> 409267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores
   [junit4]   2> 409267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoreContainer New CoreContainer 23160630
   [junit4]   2> 409267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/]
   [junit4]   2> 409267 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/lib
   [junit4]   2> 409268 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/lib).
   [junit4]   2> 409276 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 409277 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 409278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 409278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 409278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35378/solr
   [junit4]   2> 409278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 409278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 409279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 409281 INFO  (zkCallback-1583-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20fb555b name:ZooKeeperConnection Watcher:127.0.0.1:35378 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 409286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 409286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 409299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 409301 INFO  (zkCallback-1584-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f6e1ab9 name:ZooKeeperConnection Watcher:127.0.0.1:35378/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 409301 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 409302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 409304 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 409307 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 409309 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 409311 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 409312 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 409314 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 409315 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 409316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 409330 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51235_
   [junit4]   2> 409334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51235_
   [junit4]   2> 409343 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 409349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 409350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 409354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94510142302519300-127.0.0.1:51235_-n_0000000000
   [junit4]   2> 409360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51235_
   [junit4]   2> 409360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 409362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer Overseer (id=94510142302519300-127.0.0.1:51235_-n_0000000000) starting
   [junit4]   2> 409363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 409368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 409382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 409382 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 409382 INFO  (OverseerCollectionConfigSetProcessor-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 409410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 409410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 409414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores
   [junit4]   2> 409415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, config=solrconfig.xml, collection=control_collection, dataDir=data/, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1, schema=schema.xml, name=collection1, loadOnStartup=true, transient=false, coreNodeName=}
   [junit4]   2> 409415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/
   [junit4]   2> 409416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 409418 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 409418 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 409419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 409419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:51235_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 409428 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51235_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51235"} current state version: 0
   [junit4]   2> 409428 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51235_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51235"}
   [junit4]   2> 409429 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 409429 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 409431 INFO  (zkCallback-1584-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 409431 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 409432 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 409432 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 409433 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 409433 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 409434 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 409435 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/'
   [junit4]   2> 409487 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 409493 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 409506 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 409534 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 409535 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 409542 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 409790 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 409807 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 409809 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 409821 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 409828 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 409840 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 409842 WARN  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 409842 WARN  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 409842 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 409843 WARN  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 409843 WARN  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 409844 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 409844 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 409844 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 409845 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 409845 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f117668
   [junit4]   2> 409846 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/data
   [junit4]   2> 409846 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/data/index/
   [junit4]   2> 409846 WARN  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 409847 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/data/index
   [junit4]   2> 409847 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=78.5771484375, floorSegmentMB=2.107421875, forceMergeDeletesPctAllowed=0.33397045672515, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7013560273431051
   [junit4]   2> 409851 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69da79bb),segFN=segments_1,generation=1}
   [junit4]   2> 409851 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 409854 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/data/
   [junit4]   2> 409854 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 409856 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 409857 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 409857 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 409857 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 409857 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 409858 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 409858 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 409858 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 409858 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 409861 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 409863 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 409864 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 409865 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 409870 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/update/json,/admin/mbeans,/admin/file,/get,/admin/system,/update/json/docs,/replication,/update,/admin/plugins,/admin/logging,/admin/threads,/admin/segments,/admin/ping,/config,/admin/luke,/schema,/update/csv,/admin/properties
   [junit4]   2> 409871 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 409872 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 409872 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 409873 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 409873 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 409873 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=1.15625, floorSegmentMB=1.072265625, forceMergeDeletesPctAllowed=16.717151686608826, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 409875 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69da79bb),segFN=segments_1,generation=1}
   [junit4]   2> 409875 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 409876 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@36529a4c[collection1] main
   [junit4]   2> 409876 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 409877 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 409877 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 409877 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 409878 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 409878 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 409878 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 409878 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 409878 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 409879 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 409879 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 409879 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 409879 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 409879 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1512162278262177792
   [junit4]   2> 409881 INFO  (searcherExecutor-1614-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@36529a4c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 409882 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 3.0ms to seed version buckets with highest version 1512162278262177792
   [junit4]   2> 409882 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 409883 INFO  (coreLoadExecutor-1613-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 409885 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:51235 collection:control_collection shard:shard1
   [junit4]   2> 409887 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 409891 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94510142302519300-core_node1-n_0000000000
   [junit4]   2> 409892 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 409894 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 409894 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 409894 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51235/collection1/
   [junit4]   2> 409894 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 409894 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 409895 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51235/collection1/ has no replicas
   [junit4]   2> 409895 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51235/collection1/ shard1
   [junit4]   2> 409895 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 409898 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 409901 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51235",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 409920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 409935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 409937 INFO  (zkCallback-1586-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2505164 name:ZooKeeperConnection Watcher:127.0.0.1:35378/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 409937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 409937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 409937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 409942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 410003 INFO  (zkCallback-1586-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 410004 INFO  (zkCallback-1584-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 410052 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:51235/collection1/ and leader is http://127.0.0.1:51235/collection1/
   [junit4]   2> 410052 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51235
   [junit4]   2> 410052 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 410052 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 410052 INFO  (coreZkRegister-1607-thread-1-processing-s:shard1 x:collection1 n:127.0.0.1:51235_ r:core_node1 c:control_collection) [n:127.0.0.1:51235_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 410055 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51235_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51235",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 410055 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:51235_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51235",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 410158 INFO  (zkCallback-1586-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 410158 INFO  (zkCallback-1584-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 410242 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores/collection1
   [junit4]   2> 410243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001
   [junit4]   2> 410249 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 410255 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36bef22e{/,null,AVAILABLE}
   [junit4]   2> 410255 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2647e806{HTTP/1.1}{127.0.0.1:46937}
   [junit4]   2> 410255 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.e.j.s.Server Started @414032ms
   [junit4]   2> 410256 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=46937, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/tempDir-001/jetty1}
   [junit4]   2> 410256 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 410256 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/'
   [junit4]   2> 410288 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 410299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 410300 INFO  (zkCallback-1587-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5dd141d5 name:ZooKeeperConnection Watcher:127.0.0.1:35378/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 410300 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 410301 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 410314 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 410314 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/solr.xml
   [junit4]   2> 410328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores
   [junit4]   2> 410328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoreContainer New CoreContainer 802143387
   [junit4]   2> 410328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/]
   [junit4]   2> 410328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/lib
   [junit4]   2> 410328 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/lib).
   [junit4]   2> 410345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 410346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 410347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 410347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 410347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35378/solr
   [junit4]   2> 410347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 410348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 410349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 410354 INFO  (zkCallback-1589-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59bfb42f name:ZooKeeperConnection Watcher:127.0.0.1:35378 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 410355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 410356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 410362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 410380 INFO  (zkCallback-1590-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74873a72 name:ZooKeeperConnection Watcher:127.0.0.1:35378/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 410380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 410385 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 411392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46937_
   [junit4]   2> 411392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46937_
   [junit4]   2> 411394 INFO  (zkCallback-1586-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 411394 INFO  (zkCallback-1590-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 411395 INFO  (zkCallback-1584-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 411396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 411396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94510142302519304-127.0.0.1:46937_-n_0000000001
   [junit4]   2> 411397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94510142302519300-127.0.0.1:51235_-n_0000000000 to know if I could be the leader
   [junit4]   2> 411409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 411409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 411410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores
   [junit4]   2> 411411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, coreNodeName=, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores/collection1, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores/collection1/, transient=false, name=collection1, config=solrconfig.xml}
   [junit4]   2> 411411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores/collection1/
   [junit4]   2> 411411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 411413 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 411413 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 411415 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 411422 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:46937_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:46937"} current state version: 3
   [junit4]   2> 411423 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:46937_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:46937"}
   [junit4]   2> 411423 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 411423 INFO  (OverseerStateUpdate-94510142302519300-127.0.0.1:51235_-n_0000000000) [n:127.0.0.1:51235_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 411435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 411435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[331E276C7B3545BA]) [n:127.0.0.1:46937_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 411526 INFO  (zkCallback-1590-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 411526 INFO  (zkCallback-1584-thread-1-processing-n:127.0.0.1:51235_) [n:127.0.0.1:51235_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 411526 INFO  (zkCallback-1586-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 412416 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 412416 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 412418 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 412418 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 412419 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 412419 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_331E276C7B3545BA-001/shard-1-001/cores/collection1/'
   [junit4]   2> 412450 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 412462 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 412491 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 412518 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 412520 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 412526 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 412842 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 412850 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 412852 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:46937_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 412866 INFO  (coreLoadExecutor-1624-thread-1-processing-n:127.0.0.1:46937_) [n:127.0.0.1:469

[...truncated too long message...]

ave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20150912_220513_222.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20150912_220513_222.suites
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 422 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
ERROR: Failed to archive {solr/build/solr-core/test/temp/junit4-J0-20150912_220513_222.events=solr/build/solr-core/test/temp/junit4-J0-20150912_220513_222.events, README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J1-20150912_220513_222.events=solr/build/solr-core/test/temp/junit4-J1-20150912_220513_222.events, solr/build/solr-core/test/temp/junit4-J2-20150912_220513_222.events=solr/build/solr-core/test/temp/junit4-J2-20150912_220513_222.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
	at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
	at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
	at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:145)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException: unexpected EOF with 2048 bytes unread
	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:349)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 13 more
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 954 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/954/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=117101, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=117101, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38743/lh: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([B8F565A83D6FAFDB]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10652 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150911_020153_502.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18417.hprof ...
   [junit4] Heap dump file created [662118255 bytes in 5.707 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150911_020153_502.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J2: EOF ----

[...truncated 828 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/init-core-data-001
   [junit4]   2> 3752976 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 3752976 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /lh/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3753042 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3753049 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3753052 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3753066 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_57133_hdfs____.3gu6lj/webapp
   [junit4]   2> 3753179 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3753578 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57133
   [junit4]   2> 3753638 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3753639 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3753654 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_55285_datanode____n1sfw6/webapp
   [junit4]   2> 3753766 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3754246 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55285
   [junit4]   2> 3754279 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3754281 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3754296 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59317_datanode____.7zzs4k/webapp
   [junit4]   2> 3754325 INFO  (IPC Server handler 0 on 60405) [    ] BlockStateChange BLOCK* processReport: from storage DS-eeca2741-d50f-4fb8-9b0d-080fac3939be node DatanodeRegistration(127.0.0.1, datanodeUuid=7f00232a-c89c-466c-a9df-60000b254c5b, infoPort=55285, ipcPort=41461, storageInfo=lv=-56;cid=testClusterID;nsid=539081596;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3754326 INFO  (IPC Server handler 0 on 60405) [    ] BlockStateChange BLOCK* processReport: from storage DS-b9c510a1-1e56-4299-be51-a8c9462bfbbd node DatanodeRegistration(127.0.0.1, datanodeUuid=7f00232a-c89c-466c-a9df-60000b254c5b, infoPort=55285, ipcPort=41461, storageInfo=lv=-56;cid=testClusterID;nsid=539081596;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3754418 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3754848 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8F565A83D6FAFDB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59317
   [junit4]   2> 3754916 INFO  (IPC Server handler 4 on 60405) [    ] BlockStateChange BLOCK* processReport: from storage DS-0859931a-8c2e-4941-a8a2-e43ad6a2f14b node DatanodeRegistration(127.0.0.1, datanodeUuid=355f2cf2-6a02-4d77-b820-034ad1439827, infoPort=59317, ipcPort=35247, storageInfo=lv=-56;cid=testClusterID;nsid=539081596;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3754917 INFO  (IPC Server handler 4 on 60405) [    ] BlockStateChange BLOCK* processReport: from storage DS-b418047e-07b4-4cf3-af4f-a50d540909ea node DatanodeRegistration(127.0.0.1, datanodeUuid=355f2cf2-6a02-4d77-b820-034ad1439827, infoPort=59317, ipcPort=35247, storageInfo=lv=-56;cid=testClusterID;nsid=539081596;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3755033 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3755033 INFO  (Thread-106261) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3755033 INFO  (Thread-106261) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3755133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.ZkTestServer start zk server on port:35063
   [junit4]   2> 3755134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3755134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3755137 INFO  (zkCallback-1524-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78b623b9 name:ZooKeeperConnection Watcher:127.0.0.1:35063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3755137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3755137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3755137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3755140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3755141 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3755142 INFO  (zkCallback-1525-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@624685d8 name:ZooKeeperConnection Watcher:127.0.0.1:35063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3755142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3755142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3755142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3755144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3755145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3755146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3755148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3755148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3755150 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3755151 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3755152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3755153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3755154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3755154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3755156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3755156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3755158 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3755158 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3755159 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3755160 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3755161 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3755161 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3755163 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3755163 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3755165 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3755165 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3755166 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3755166 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3755444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores/collection1
   [junit4]   2> 3755446 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3755448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7ccae7a2{/lh,null,AVAILABLE}
   [junit4]   2> 3755448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d2d7caf{HTTP/1.1}{127.0.0.1:55461}
   [junit4]   2> 3755448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.Server Started @3758752ms
   [junit4]   2> 3755448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores, hostPort=55461, hostContext=/lh, solr.data.dir=hdfs://localhost:60405/hdfs__localhost_60405__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001_tempDir-002_control_data}
   [junit4]   2> 3755449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3755449 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/'
   [junit4]   2> 3755464 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3755465 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3755486 INFO  (zkCallback-1526-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f79160c name:ZooKeeperConnection Watcher:127.0.0.1:35063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3755488 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3755488 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3755490 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3755491 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/solr.xml
   [junit4]   2> 3755504 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores
   [junit4]   2> 3755505 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoreContainer New CoreContainer 422377666
   [junit4]   2> 3755505 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/]
   [junit4]   2> 3755505 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/lib
   [junit4]   2> 3755505 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/lib).
   [junit4]   2> 3755513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3755514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3755515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3755515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3755515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35063/solr
   [junit4]   2> 3755515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3755515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3755516 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3755517 INFO  (zkCallback-1528-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37ab6d07 name:ZooKeeperConnection Watcher:127.0.0.1:35063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3755517 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3755518 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3755519 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3755520 INFO  (zkCallback-1529-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f378ebc name:ZooKeeperConnection Watcher:127.0.0.1:35063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3755520 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3755521 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3755523 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3755526 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3755528 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3755530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3755532 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3755534 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3755535 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3755536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3755537 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55461_lh
   [junit4]   2> 3755538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55461_lh
   [junit4]   2> 3755539 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3755541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3755542 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3755542 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94499967595905028-127.0.0.1:55461_lh-n_0000000000
   [junit4]   2> 3755543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55461_lh
   [junit4]   2> 3755543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3755544 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer Overseer (id=94499967595905028-127.0.0.1:55461_lh-n_0000000000) starting
   [junit4]   2> 3755546 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3755551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3755552 INFO  (OverseerCollectionConfigSetProcessor-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3755552 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3755552 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3755565 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3755565 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3755566 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores
   [junit4]   2> 3755567 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=control_collection, dataDir=data/, schema=schema.xml, name=collection1, config=solrconfig.xml, transient=false, configSetProperties=configsetprops.json, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores/collection1/, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores/collection1, shard=}
   [junit4]   2> 3755567 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores/collection1/
   [junit4]   2> 3755567 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3755568 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3755568 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55461_lh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3755568 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3755569 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3755571 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:55461_lh",
   [junit4]   2>   "base_url":"http://127.0.0.1:55461/lh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 3755572 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3755572 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:55461_lh",
   [junit4]   2>   "base_url":"http://127.0.0.1:55461/lh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3755572 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3755573 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3755574 INFO  (zkCallback-1529-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3756572 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3756573 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3756573 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3756573 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3756574 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3756574 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores/collection1/'
   [junit4]   2> 3756595 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3756600 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3756613 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3756641 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3756642 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3756649 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3757752 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3757761 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3757763 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3757772 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3757775 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3757777 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3757778 WARN  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3757778 WARN  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3757778 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3757779 WARN  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3757779 WARN  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3757780 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3757780 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3757780 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3757781 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:60405/solr_hdfs_home
   [junit4]   2> 3757781 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3757781 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3757781 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@327e9516
   [junit4]   2> 3757782 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3757797 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 3757797 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3757798 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3757808 WARN  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:60405/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3757808 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3757819 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 3757819 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3757820 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=12, maxMergedSegmentMB=0.9560546875, floorSegmentMB=1.6943359375, forceMergeDeletesPctAllowed=27.97145781280398, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 3757833 INFO  (IPC Server handler 4 on 60405) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35528 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-b9c510a1-1e56-4299-be51-a8c9462bfbbd:NORMAL:127.0.0.1:53402|RBW], ReplicaUnderConstruction[[DISK]DS-0859931a-8c2e-4941-a8a2-e43ad6a2f14b:NORMAL:127.0.0.1:35528|RBW]]} size 0
   [junit4]   2> 3757834 INFO  (IPC Server handler 0 on 60405) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:53402 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-0859931a-8c2e-4941-a8a2-e43ad6a2f14b:NORMAL:127.0.0.1:35528|RBW], ReplicaUnderConstruction[[DISK]DS-eeca2741-d50f-4fb8-9b0d-080fac3939be:NORMAL:127.0.0.1:53402|FINALIZED]]} size 0
   [junit4]   2> 3757839 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92db8b7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@647720e; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3757840 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3757841 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:60405/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3757844 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3757845 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3757845 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3757845 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3757845 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3757845 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3757846 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3757846 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3757846 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3757849 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3757850 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3757851 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3757853 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3757856 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/mbeans,/update/csv,/admin/system,/replication,/get,/update/json,/update/json/docs,/admin/properties,/admin/logging,/update,/admin/ping,/admin/threads,/admin/plugins,/admin/luke,/config,/admin/segments,/schema,/admin/file
   [junit4]   2> 3757858 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3757859 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3757859 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 3757870 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3757870 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3757871 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=1.73046875, floorSegmentMB=0.3671875, forceMergeDeletesPctAllowed=26.258917654448606, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8058235739273079
   [junit4]   2> 3757875 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92db8b7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@647720e; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3757875 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3757876 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@26ffa7f6[collection1] main
   [junit4]   2> 3757876 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3757877 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3757877 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3757878 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3757878 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3757878 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3757878 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3757878 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3757879 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3757879 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3757880 INFO  (searcherExecutor-5693-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@26ffa7f6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3757881 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3757881 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 3757881 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3757881 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511999484518727680
   [junit4]   2> 3757881 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1511999484518727680
   [junit4]   2> 3757882 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3757882 INFO  (coreLoadExecutor-5692-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3757882 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:55461/lh collection:control_collection shard:shard1
   [junit4]   2> 3757883 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3757886 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94499967595905028-core_node1-n_0000000000
   [junit4]   2> 3757887 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3757900 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3757900 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3757900 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3757901 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55461/lh/collection1/
   [junit4]   2> 3757901 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3757901 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55461/lh/collection1/ has no replicas
   [junit4]   2> 3757901 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55461/lh/collection1/ shard1
   [junit4]   2> 3757901 INFO  (zkCallback-1529-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3757902 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3757905 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3757907 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:55461/lh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 3758010 INFO  (zkCallback-1529-thread-1-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3758058 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:55461/lh/collection1/ and leader is http://127.0.0.1:55461/lh/collection1/
   [junit4]   2> 3758058 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55461/lh
   [junit4]   2> 3758058 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3758058 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3758058 INFO  (coreZkRegister-5686-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:55461_lh x:collection1) [n:127.0.0.1:55461_lh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3758061 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:55461_lh",
   [junit4]   2>   "base_url":"http://127.0.0.1:55461/lh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null} current state version: 3
   [junit4]   2> 3758062 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:55461_lh",
   [junit4]   2>   "base_url":"http://127.0.0.1:55461/lh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3758094 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3758095 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3758096 INFO  (zkCallback-1531-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@440db0e name:ZooKeeperConnection Watcher:127.0.0.1:35063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3758096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3758096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3758097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3758099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3758164 INFO  (zkCallback-1531-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3758164 INFO  (zkCallback-1529-thread-2-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3758394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores/collection1
   [junit4]   2> 3758395 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001
   [junit4]   2> 3758396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3758399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7478c061{/lh,null,AVAILABLE}
   [junit4]   2> 3758399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@71ab1ce0{HTTP/1.1}{127.0.0.1:55400}
   [junit4]   2> 3758399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.e.j.s.Server Started @3761703ms
   [junit4]   2> 3758400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores, hostContext=/lh, hostPort=55400, solr.data.dir=hdfs://localhost:60405/hdfs__localhost_60405__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001_tempDir-002_jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 3758400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3758400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/'
   [junit4]   2> 3758415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3758416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3758417 INFO  (zkCallback-1532-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5918ec4a name:ZooKeeperConnection Watcher:127.0.0.1:35063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3758417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3758418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3758419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3758419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/solr.xml
   [junit4]   2> 3758433 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores
   [junit4]   2> 3758433 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoreContainer New CoreContainer 583475340
   [junit4]   2> 3758433 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/]
   [junit4]   2> 3758433 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/lib
   [junit4]   2> 3758433 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/lib).
   [junit4]   2> 3758441 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3758442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3758443 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3758443 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3758443 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35063/solr
   [junit4]   2> 3758443 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3758443 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3758444 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3758445 INFO  (zkCallback-1534-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79b45bdf name:ZooKeeperConnection Watcher:127.0.0.1:35063 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3758445 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3758446 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3758447 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3758448 INFO  (zkCallback-1535-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@583cc6b6 name:ZooKeeperConnection Watcher:127.0.0.1:35063/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3758448 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3758453 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3759458 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55400_lh
   [junit4]   2> 3759458 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55400_lh
   [junit4]   2> 3759460 INFO  (zkCallback-1531-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3759460 INFO  (zkCallback-1529-thread-2-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3759460 INFO  (zkCallback-1535-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3759461 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3759461 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94499967595905032-127.0.0.1:55400_lh-n_0000000001
   [junit4]   2> 3759462 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94499967595905028-127.0.0.1:55461_lh-n_0000000000 to know if I could be the leader
   [junit4]   2> 3759478 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3759479 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3759480 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores
   [junit4]   2> 3759481 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, configSetProperties=configsetprops.json, transient=false, loadOnStartup=true, config=solrconfig.xml, shard=, coreNodeName=, collection=collection1, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores/collection1/, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores/collection1}
   [junit4]   2> 3759481 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores/collection1/
   [junit4]   2> 3759482 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3759483 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3759483 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B8F565A83D6FAFDB]) [n:127.0.0.1:55400_lh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3759484 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3759484 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3759486 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:55400_lh",
   [junit4]   2>   "base_url":"http://127.0.0.1:55400/lh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null} current state version: 4
   [junit4]   2> 3759487 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3759487 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:55400_lh",
   [junit4]   2>   "base_url":"http://127.0.0.1:55400/lh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3759487 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3759487 INFO  (OverseerStateUpdate-94499967595905028-127.0.0.1:55461_lh-n_0000000000) [n:127.0.0.1:55461_lh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 3759590 INFO  (zkCallback-1529-thread-2-processing-n:127.0.0.1:55461_lh) [n:127.0.0.1:55461_lh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3759590 INFO  (zkCallback-1535-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3759590 INFO  (zkCallback-1531-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3760487 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3760488 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 3760489 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3760489 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 3760489 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3760489 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8F565A83D6FAFDB-001/shard-1-001/cores/collection1/'
   [junit4]   2> 3760506 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3760513 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3760527 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3760555 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3760556 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3760563 INFO  (coreLoadExecutor-5703-thread-1-processing-n:127.0.0.1:55400_lh) [n:127.0.0.1:55400_lh c:collection1   x:coll

[...truncated too long message...]

ce/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150911_020153_502.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150911_020153_502.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 206 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1852837366 bytes
Compression is 0.0%
Took 5 min 32 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing

Posted by Robert Muir <rc...@gmail.com>.
its a static leak in Nightly test. I will fix it.

On Thu, Sep 10, 2015 at 5:59 AM, Dawid Weiss <da...@gmail.com> wrote:
> RamUsageEstimator tries to measure something that is doesn't have
> access to, huh?
>
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>         at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
>         at java.security.AccessController.checkPermission(AccessController.java:559)
>         at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>         at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>         at java.lang.Class.checkPackageAccess(Class.java:2309)
>         at java.lang.Class.checkMemberAccess(Class.java:2289)
>         at java.lang.Class.getDeclaredFields(Class.java:1810)
>         at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:573)
>
> On Thu, Sep 10, 2015 at 11:49 AM, Apache Jenkins Server
> <je...@builds.apache.org> wrote:
>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/953/
>>
>> 1 tests failed.
>> FAILED:  junit.framework.TestSuite.org.apache.lucene.index.IndexSortingTest
>>
>> Error Message:
>> access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>>
>> Stack Trace:
>> java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>>         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>>         at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
>>         at java.security.AccessController.checkPermission(AccessController.java:559)
>>         at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>>         at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>>         at java.lang.Class.checkPackageAccess(Class.java:2309)
>>         at java.lang.Class.checkMemberAccess(Class.java:2289)
>>         at java.lang.Class.getDeclaredFields(Class.java:1810)
>>         at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:573)
>>         at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSizeOf(RamUsageEstimator.java:537)
>>         at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(RamUsageEstimator.java:385)
>>         at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:108)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>         at java.lang.Thread.run(Thread.java:745)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 8052 lines...]
>>    [junit4] Suite: org.apache.lucene.index.IndexSortingTest
>>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {term_vectors=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=MockRandom), positions=FSTOrd50, docs=PostingsFormat(name=MockRandom), norm=PostingsFormat(name=Memory doPackFST= true)}, docValues:{sorted_set=DocValuesFormat(name=Memory), numeric=DocValuesFormat(name=Lucene50), binary=DocValuesFormat(name=Lucene50), sorted_numeric=DocValuesFormat(name=Asserting), sorted=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=be_BY, timezone=America/Los_Angeles
>>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=188195696,total=326107136
>>    [junit4]   2> NOTE: All tests run in this JVM: [TestFieldCacheReopen, TestFieldCacheWithThreads, TestFieldCacheSortRandom, TestNumericTerms32, TestIndexSplitter, TestPKIndexSplitter, TestMultiPassIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest, TestLazyDocument, IndexSortingTest]
>>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSortingTest -Dtests.seed=4146977D8265D175 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=be_BY -Dtests.timezone=America/Los_Angeles -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>>    [junit4] ERROR   0.00s J1 | IndexSortingTest (suite) <<<
>>    [junit4]    > Throwable #1: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>>    [junit4]    >        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
>>    [junit4]    >        at java.security.AccessController.checkPermission(AccessController.java:559)
>>    [junit4]    >        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>>    [junit4]    >        at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>>    [junit4]    >        at java.lang.Class.checkPackageAccess(Class.java:2309)
>>    [junit4]    >        at java.lang.Class.checkMemberAccess(Class.java:2289)
>>    [junit4]    >        at java.lang.Class.getDeclaredFields(Class.java:1810)
>>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>>    [junit4] Completed [14/25] on J1 in 1.09s, 9 tests, 1 error <<< FAILURES!
>>
>> [...truncated 47 lines...]
>> BUILD FAILED
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 25 suites, 184 tests, 1 suite-level error, 1 ignored (1 assumption)
>>
>> Total time: 249 minutes 13 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
>> Archived 4 artifacts
>> Archive block size is 32768
>> Received 0 blocks and 232124 bytes
>> Compression is 0.0%
>> Took 0.17 sec
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing

Posted by Robert Muir <rc...@gmail.com>.
I am also happy to submit improvements to RR for this as well. It
doesnt matter, its just a matter of time. I was blocked from working
on it before by the insanity known as maven
(https://github.com/randomizedtesting/randomizedtesting/issues/199)
but now I can get past the issues.

On Thu, Sep 10, 2015 at 2:53 PM, Dawid Weiss <da...@gmail.com> wrote:
> I told Robert in a private conversation that I think the RUE rule
> should be copied to Lucene and tweaked in here (where the security
> manager is present for tests and where there's so much testing against
> new JVMs). I'll gladly port the changes back to the RR project, it's
> justy for convenience that I think we should have a Lucene copy.
>
> Dawid
>
> On Thu, Sep 10, 2015 at 3:32 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
>> Yes,
>>
>> I think because the error message is very confusing, maybe RamUsageEstimator should catch this exception and then complain with "Class leaks a static instance of XXXX with unknown size."
>> This would make it easier for developers to figure out what's wrong.
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>>> -----Original Message-----
>>> From: Dawid Weiss [mailto:dawid.weiss@gmail.com]
>>> Sent: Thursday, September 10, 2015 12:00 PM
>>> To: dev@lucene.apache.org
>>> Subject: Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing
>>>
>>> RamUsageEstimator tries to measure something that is doesn't have access
>>> to, huh?
>>>
>>> java.security.AccessControlException: access denied
>>> ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>>>         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>>>         at
>>> java.security.AccessControlContext.checkPermission(AccessControlContext.j
>>> ava:372)
>>>         at
>>> java.security.AccessController.checkPermission(AccessController.java:559)
>>>         at
>>> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>>>         at
>>> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>>>         at java.lang.Class.checkPackageAccess(Class.java:2309)
>>>         at java.lang.Class.checkMemberAccess(Class.java:2289)
>>>         at java.lang.Class.getDeclaredFields(Class.java:1810)
>>>         at
>>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCache
>>> Entry(RamUsageEstimator.java:573)
>>>
>>> On Thu, Sep 10, 2015 at 11:49 AM, Apache Jenkins Server
>>> <je...@builds.apache.org> wrote:
>>> > Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/953/
>>> >
>>> > 1 tests failed.
>>> > FAILED:
>>> > junit.framework.TestSuite.org.apache.lucene.index.IndexSortingTest
>>> >
>>> > Error Message:
>>> > access denied ("java.lang.RuntimePermission"
>>> > "accessClassInPackage.sun.nio.ch")
>>> >
>>> > Stack Trace:
>>> > java.security.AccessControlException: access denied
>>> ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>>> >         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>>> >         at
>>> java.security.AccessControlContext.checkPermission(AccessControlContext.j
>>> ava:372)
>>> >         at
>>> java.security.AccessController.checkPermission(AccessController.java:559)
>>> >         at
>>> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>>> >         at
>>> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>>> >         at java.lang.Class.checkPackageAccess(Class.java:2309)
>>> >         at java.lang.Class.checkMemberAccess(Class.java:2289)
>>> >         at java.lang.Class.getDeclaredFields(Class.java:1810)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCache
>>> Entry(RamUsageEstimator.java:573)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSize
>>> Of(RamUsageEstimator.java:537)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(Ra
>>> mUsageEstimator.java:385)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterA
>>> lways(StaticFieldsInvariantRule.java:108)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>>> ementAdapter.java:43)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>>> ementAdapter.java:36)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>>> ementAdapter.java:36)
>>> >         at
>>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss
>>> ertionsRequired.java:54)
>>> >         at
>>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure
>>> .java:48)
>>> >         at
>>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule
>>> IgnoreAfterMaxFailures.java:65)
>>> >         at
>>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore
>>> TestSuites.java:55)
>>> >         at
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>>> ementAdapter.java:36)
>>> >         at
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
>>> run(ThreadLeakControl.java:365)
>>> >         at java.lang.Thread.run(Thread.java:745)
>>> >
>>> >
>>> >
>>> >
>>> > Build Log:
>>> > [...truncated 8052 lines...]
>>> >    [junit4] Suite: org.apache.lucene.index.IndexSortingTest
>>> >    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53):
>>> {term_vectors=PostingsFormat(name=LuceneVarGapDocFreqInterval),
>>> id=PostingsFormat(name=MockRandom), positions=FSTOrd50,
>>> docs=PostingsFormat(name=MockRandom),
>>> norm=PostingsFormat(name=Memory doPackFST= true)},
>>> docValues:{sorted_set=DocValuesFormat(name=Memory),
>>> numeric=DocValuesFormat(name=Lucene50),
>>> binary=DocValuesFormat(name=Lucene50),
>>> sorted_numeric=DocValuesFormat(name=Asserting),
>>> sorted=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity,
>>> locale=be_BY, timezone=America/Los_Angeles
>>> >    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
>>> 1.7.0_72 (64-bit)/cpus=4,threads=1,free=188195696,total=326107136
>>> >    [junit4]   2> NOTE: All tests run in this JVM: [TestFieldCacheReopen,
>>> TestFieldCacheWithThreads, TestFieldCacheSortRandom,
>>> TestNumericTerms32, TestIndexSplitter, TestPKIndexSplitter,
>>> TestMultiPassIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest,
>>> TestLazyDocument, IndexSortingTest]
>>> >    [junit4]   2> NOTE: download the large Jenkins line-docs file by running
>>> 'ant get-jenkins-line-docs' in the lucene directory.
>>> >    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSortingTest
>>> -Dtests.seed=4146977D8265D175 -Dtests.multiplier=2 -Dtests.nightly=true -
>>> Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-
>>> data/enwiki.random.lines.txt -Dtests.locale=be_BY -
>>> Dtests.timezone=America/Los_Angeles -Dtests.asserts=true -
>>> Dtests.file.encoding=ISO-8859-1
>>> >    [junit4] ERROR   0.00s J1 | IndexSortingTest (suite) <<<
>>> >    [junit4]    > Throwable #1: java.security.AccessControlException: access
>>> denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>>> >    [junit4]    >        at
>>> __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>>> >    [junit4]    >        at
>>> java.security.AccessControlContext.checkPermission(AccessControlContext.j
>>> ava:372)
>>> >    [junit4]    >        at
>>> java.security.AccessController.checkPermission(AccessController.java:559)
>>> >    [junit4]    >        at
>>> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>>> >    [junit4]    >        at
>>> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>>> >    [junit4]    >        at java.lang.Class.checkPackageAccess(Class.java:2309)
>>> >    [junit4]    >        at java.lang.Class.checkMemberAccess(Class.java:2289)
>>> >    [junit4]    >        at java.lang.Class.getDeclaredFields(Class.java:1810)
>>> >    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>>> >    [junit4] Completed [14/25] on J1 in 1.09s, 9 tests, 1 error <<< FAILURES!
>>> >
>>> > [...truncated 47 lines...]
>>> > BUILD FAILED
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>>> 5.x/build.xml:792: The following error occurred while executing this line:
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>>> 5.x/build.xml:729: The following error occurred while executing this line:
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>>> 5.x/build.xml:59: The following error occurred while executing this line:
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>>> 5.x/lucene/build.xml:471: The following error occurred while executing this
>>> line:
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>>> 5.x/lucene/common-build.xml:2248: The following error occurred while
>>> executing this line:
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>>> 5.x/lucene/module-build.xml:58: The following error occurred while
>>> executing this line:
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>>> 5.x/lucene/common-build.xml:1452: The following error occurred while
>>> executing this line:
>>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucen
>>> > e/common-build.xml:1006: There were test failures: 25 suites, 184
>>> > tests, 1 suite-level error, 1 ignored (1 assumption)
>>> >
>>> > Total time: 249 minutes 13 seconds
>>> > Build step 'Invoke Ant' marked build as failure Archiving artifacts
>>> > Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
>>> > Archived 4 artifacts Archive block size is 32768 Received 0 blocks and
>>> > 232124 bytes Compression is 0.0% Took 0.17 sec Recording test results
>>> > Email was triggered for: Failure Sending email for trigger: Failure
>>> >
>>> >
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>> > additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>>> commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing

Posted by Dawid Weiss <da...@gmail.com>.
I told Robert in a private conversation that I think the RUE rule
should be copied to Lucene and tweaked in here (where the security
manager is present for tests and where there's so much testing against
new JVMs). I'll gladly port the changes back to the RR project, it's
justy for convenience that I think we should have a Lucene copy.

Dawid

On Thu, Sep 10, 2015 at 3:32 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Yes,
>
> I think because the error message is very confusing, maybe RamUsageEstimator should catch this exception and then complain with "Class leaks a static instance of XXXX with unknown size."
> This would make it easier for developers to figure out what's wrong.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Dawid Weiss [mailto:dawid.weiss@gmail.com]
>> Sent: Thursday, September 10, 2015 12:00 PM
>> To: dev@lucene.apache.org
>> Subject: Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing
>>
>> RamUsageEstimator tries to measure something that is doesn't have access
>> to, huh?
>>
>> java.security.AccessControlException: access denied
>> ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>>         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>>         at
>> java.security.AccessControlContext.checkPermission(AccessControlContext.j
>> ava:372)
>>         at
>> java.security.AccessController.checkPermission(AccessController.java:559)
>>         at
>> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>>         at
>> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>>         at java.lang.Class.checkPackageAccess(Class.java:2309)
>>         at java.lang.Class.checkMemberAccess(Class.java:2289)
>>         at java.lang.Class.getDeclaredFields(Class.java:1810)
>>         at
>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCache
>> Entry(RamUsageEstimator.java:573)
>>
>> On Thu, Sep 10, 2015 at 11:49 AM, Apache Jenkins Server
>> <je...@builds.apache.org> wrote:
>> > Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/953/
>> >
>> > 1 tests failed.
>> > FAILED:
>> > junit.framework.TestSuite.org.apache.lucene.index.IndexSortingTest
>> >
>> > Error Message:
>> > access denied ("java.lang.RuntimePermission"
>> > "accessClassInPackage.sun.nio.ch")
>> >
>> > Stack Trace:
>> > java.security.AccessControlException: access denied
>> ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>> >         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>> >         at
>> java.security.AccessControlContext.checkPermission(AccessControlContext.j
>> ava:372)
>> >         at
>> java.security.AccessController.checkPermission(AccessController.java:559)
>> >         at
>> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>> >         at
>> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>> >         at java.lang.Class.checkPackageAccess(Class.java:2309)
>> >         at java.lang.Class.checkMemberAccess(Class.java:2289)
>> >         at java.lang.Class.getDeclaredFields(Class.java:1810)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCache
>> Entry(RamUsageEstimator.java:573)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSize
>> Of(RamUsageEstimator.java:537)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(Ra
>> mUsageEstimator.java:385)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterA
>> lways(StaticFieldsInvariantRule.java:108)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>> ementAdapter.java:43)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>> ementAdapter.java:36)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>> ementAdapter.java:36)
>> >         at
>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss
>> ertionsRequired.java:54)
>> >         at
>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure
>> .java:48)
>> >         at
>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule
>> IgnoreAfterMaxFailures.java:65)
>> >         at
>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore
>> TestSuites.java:55)
>> >         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
>> ementAdapter.java:36)
>> >         at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
>> run(ThreadLeakControl.java:365)
>> >         at java.lang.Thread.run(Thread.java:745)
>> >
>> >
>> >
>> >
>> > Build Log:
>> > [...truncated 8052 lines...]
>> >    [junit4] Suite: org.apache.lucene.index.IndexSortingTest
>> >    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53):
>> {term_vectors=PostingsFormat(name=LuceneVarGapDocFreqInterval),
>> id=PostingsFormat(name=MockRandom), positions=FSTOrd50,
>> docs=PostingsFormat(name=MockRandom),
>> norm=PostingsFormat(name=Memory doPackFST= true)},
>> docValues:{sorted_set=DocValuesFormat(name=Memory),
>> numeric=DocValuesFormat(name=Lucene50),
>> binary=DocValuesFormat(name=Lucene50),
>> sorted_numeric=DocValuesFormat(name=Asserting),
>> sorted=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity,
>> locale=be_BY, timezone=America/Los_Angeles
>> >    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
>> 1.7.0_72 (64-bit)/cpus=4,threads=1,free=188195696,total=326107136
>> >    [junit4]   2> NOTE: All tests run in this JVM: [TestFieldCacheReopen,
>> TestFieldCacheWithThreads, TestFieldCacheSortRandom,
>> TestNumericTerms32, TestIndexSplitter, TestPKIndexSplitter,
>> TestMultiPassIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest,
>> TestLazyDocument, IndexSortingTest]
>> >    [junit4]   2> NOTE: download the large Jenkins line-docs file by running
>> 'ant get-jenkins-line-docs' in the lucene directory.
>> >    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSortingTest
>> -Dtests.seed=4146977D8265D175 -Dtests.multiplier=2 -Dtests.nightly=true -
>> Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-
>> data/enwiki.random.lines.txt -Dtests.locale=be_BY -
>> Dtests.timezone=America/Los_Angeles -Dtests.asserts=true -
>> Dtests.file.encoding=ISO-8859-1
>> >    [junit4] ERROR   0.00s J1 | IndexSortingTest (suite) <<<
>> >    [junit4]    > Throwable #1: java.security.AccessControlException: access
>> denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>> >    [junit4]    >        at
>> __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>> >    [junit4]    >        at
>> java.security.AccessControlContext.checkPermission(AccessControlContext.j
>> ava:372)
>> >    [junit4]    >        at
>> java.security.AccessController.checkPermission(AccessController.java:559)
>> >    [junit4]    >        at
>> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>> >    [junit4]    >        at
>> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>> >    [junit4]    >        at java.lang.Class.checkPackageAccess(Class.java:2309)
>> >    [junit4]    >        at java.lang.Class.checkMemberAccess(Class.java:2289)
>> >    [junit4]    >        at java.lang.Class.getDeclaredFields(Class.java:1810)
>> >    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>> >    [junit4] Completed [14/25] on J1 in 1.09s, 9 tests, 1 error <<< FAILURES!
>> >
>> > [...truncated 47 lines...]
>> > BUILD FAILED
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>> 5.x/build.xml:792: The following error occurred while executing this line:
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>> 5.x/build.xml:729: The following error occurred while executing this line:
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>> 5.x/build.xml:59: The following error occurred while executing this line:
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>> 5.x/lucene/build.xml:471: The following error occurred while executing this
>> line:
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>> 5.x/lucene/common-build.xml:2248: The following error occurred while
>> executing this line:
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>> 5.x/lucene/module-build.xml:58: The following error occurred while
>> executing this line:
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
>> 5.x/lucene/common-build.xml:1452: The following error occurred while
>> executing this line:
>> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucen
>> > e/common-build.xml:1006: There were test failures: 25 suites, 184
>> > tests, 1 suite-level error, 1 ignored (1 assumption)
>> >
>> > Total time: 249 minutes 13 seconds
>> > Build step 'Invoke Ant' marked build as failure Archiving artifacts
>> > Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
>> > Archived 4 artifacts Archive block size is 32768 Received 0 blocks and
>> > 232124 bytes Compression is 0.0% Took 0.17 sec Recording test results
>> > Email was triggered for: Failure Sending email for trigger: Failure
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> > additional commands, e-mail: dev-help@lucene.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing

Posted by Uwe Schindler <uw...@thetaphi.de>.
Yes,

I think because the error message is very confusing, maybe RamUsageEstimator should catch this exception and then complain with "Class leaks a static instance of XXXX with unknown size."
This would make it easier for developers to figure out what's wrong.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Dawid Weiss [mailto:dawid.weiss@gmail.com]
> Sent: Thursday, September 10, 2015 12:00 PM
> To: dev@lucene.apache.org
> Subject: Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing
> 
> RamUsageEstimator tries to measure something that is doesn't have access
> to, huh?
> 
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>         at
> java.security.AccessControlContext.checkPermission(AccessControlContext.j
> ava:372)
>         at
> java.security.AccessController.checkPermission(AccessController.java:559)
>         at
> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>         at
> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>         at java.lang.Class.checkPackageAccess(Class.java:2309)
>         at java.lang.Class.checkMemberAccess(Class.java:2289)
>         at java.lang.Class.getDeclaredFields(Class.java:1810)
>         at
> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCache
> Entry(RamUsageEstimator.java:573)
> 
> On Thu, Sep 10, 2015 at 11:49 AM, Apache Jenkins Server
> <je...@builds.apache.org> wrote:
> > Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/953/
> >
> > 1 tests failed.
> > FAILED:
> > junit.framework.TestSuite.org.apache.lucene.index.IndexSortingTest
> >
> > Error Message:
> > access denied ("java.lang.RuntimePermission"
> > "accessClassInPackage.sun.nio.ch")
> >
> > Stack Trace:
> > java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
> >         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
> >         at
> java.security.AccessControlContext.checkPermission(AccessControlContext.j
> ava:372)
> >         at
> java.security.AccessController.checkPermission(AccessController.java:559)
> >         at
> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> >         at
> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
> >         at java.lang.Class.checkPackageAccess(Class.java:2309)
> >         at java.lang.Class.checkMemberAccess(Class.java:2289)
> >         at java.lang.Class.getDeclaredFields(Class.java:1810)
> >         at
> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCache
> Entry(RamUsageEstimator.java:573)
> >         at
> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSize
> Of(RamUsageEstimator.java:537)
> >         at
> com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(Ra
> mUsageEstimator.java:385)
> >         at
> com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterA
> lways(StaticFieldsInvariantRule.java:108)
> >         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:43)
> >         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> >         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> >         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss
> ertionsRequired.java:54)
> >         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure
> .java:48)
> >         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule
> IgnoreAfterMaxFailures.java:65)
> >         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore
> TestSuites.java:55)
> >         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> >         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
> run(ThreadLeakControl.java:365)
> >         at java.lang.Thread.run(Thread.java:745)
> >
> >
> >
> >
> > Build Log:
> > [...truncated 8052 lines...]
> >    [junit4] Suite: org.apache.lucene.index.IndexSortingTest
> >    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53):
> {term_vectors=PostingsFormat(name=LuceneVarGapDocFreqInterval),
> id=PostingsFormat(name=MockRandom), positions=FSTOrd50,
> docs=PostingsFormat(name=MockRandom),
> norm=PostingsFormat(name=Memory doPackFST= true)},
> docValues:{sorted_set=DocValuesFormat(name=Memory),
> numeric=DocValuesFormat(name=Lucene50),
> binary=DocValuesFormat(name=Lucene50),
> sorted_numeric=DocValuesFormat(name=Asserting),
> sorted=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity,
> locale=be_BY, timezone=America/Los_Angeles
> >    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
> 1.7.0_72 (64-bit)/cpus=4,threads=1,free=188195696,total=326107136
> >    [junit4]   2> NOTE: All tests run in this JVM: [TestFieldCacheReopen,
> TestFieldCacheWithThreads, TestFieldCacheSortRandom,
> TestNumericTerms32, TestIndexSplitter, TestPKIndexSplitter,
> TestMultiPassIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest,
> TestLazyDocument, IndexSortingTest]
> >    [junit4]   2> NOTE: download the large Jenkins line-docs file by running
> 'ant get-jenkins-line-docs' in the lucene directory.
> >    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSortingTest
> -Dtests.seed=4146977D8265D175 -Dtests.multiplier=2 -Dtests.nightly=true -
> Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-
> data/enwiki.random.lines.txt -Dtests.locale=be_BY -
> Dtests.timezone=America/Los_Angeles -Dtests.asserts=true -
> Dtests.file.encoding=ISO-8859-1
> >    [junit4] ERROR   0.00s J1 | IndexSortingTest (suite) <<<
> >    [junit4]    > Throwable #1: java.security.AccessControlException: access
> denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
> >    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
> >    [junit4]    >        at
> java.security.AccessControlContext.checkPermission(AccessControlContext.j
> ava:372)
> >    [junit4]    >        at
> java.security.AccessController.checkPermission(AccessController.java:559)
> >    [junit4]    >        at
> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> >    [junit4]    >        at
> java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
> >    [junit4]    >        at java.lang.Class.checkPackageAccess(Class.java:2309)
> >    [junit4]    >        at java.lang.Class.checkMemberAccess(Class.java:2289)
> >    [junit4]    >        at java.lang.Class.getDeclaredFields(Class.java:1810)
> >    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
> >    [junit4] Completed [14/25] on J1 in 1.09s, 9 tests, 1 error <<< FAILURES!
> >
> > [...truncated 47 lines...]
> > BUILD FAILED
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
> 5.x/build.xml:792: The following error occurred while executing this line:
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
> 5.x/build.xml:729: The following error occurred while executing this line:
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
> 5.x/build.xml:59: The following error occurred while executing this line:
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
> 5.x/lucene/build.xml:471: The following error occurred while executing this
> line:
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
> 5.x/lucene/common-build.xml:2248: The following error occurred while
> executing this line:
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
> 5.x/lucene/module-build.xml:58: The following error occurred while
> executing this line:
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-
> 5.x/lucene/common-build.xml:1452: The following error occurred while
> executing this line:
> > /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucen
> > e/common-build.xml:1006: There were test failures: 25 suites, 184
> > tests, 1 suite-level error, 1 ignored (1 assumption)
> >
> > Total time: 249 minutes 13 seconds
> > Build step 'Invoke Ant' marked build as failure Archiving artifacts
> > Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
> > Archived 4 artifacts Archive block size is 32768 Received 0 blocks and
> > 232124 bytes Compression is 0.0% Took 0.17 sec Recording test results
> > Email was triggered for: Failure Sending email for trigger: Failure
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing

Posted by Dawid Weiss <da...@gmail.com>.
RamUsageEstimator tries to measure something that is doesn't have
access to, huh?

java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
        at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
        at java.security.AccessController.checkPermission(AccessController.java:559)
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
        at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
        at java.lang.Class.checkPackageAccess(Class.java:2309)
        at java.lang.Class.checkMemberAccess(Class.java:2289)
        at java.lang.Class.getDeclaredFields(Class.java:1810)
        at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:573)

On Thu, Sep 10, 2015 at 11:49 AM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/953/
>
> 1 tests failed.
> FAILED:  junit.framework.TestSuite.org.apache.lucene.index.IndexSortingTest
>
> Error Message:
> access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>
> Stack Trace:
> java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>         at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>         at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
>         at java.security.AccessController.checkPermission(AccessController.java:559)
>         at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>         at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>         at java.lang.Class.checkPackageAccess(Class.java:2309)
>         at java.lang.Class.checkMemberAccess(Class.java:2289)
>         at java.lang.Class.getDeclaredFields(Class.java:1810)
>         at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:573)
>         at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSizeOf(RamUsageEstimator.java:537)
>         at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(RamUsageEstimator.java:385)
>         at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:108)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 8052 lines...]
>    [junit4] Suite: org.apache.lucene.index.IndexSortingTest
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {term_vectors=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=MockRandom), positions=FSTOrd50, docs=PostingsFormat(name=MockRandom), norm=PostingsFormat(name=Memory doPackFST= true)}, docValues:{sorted_set=DocValuesFormat(name=Memory), numeric=DocValuesFormat(name=Lucene50), binary=DocValuesFormat(name=Lucene50), sorted_numeric=DocValuesFormat(name=Asserting), sorted=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=be_BY, timezone=America/Los_Angeles
>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=188195696,total=326107136
>    [junit4]   2> NOTE: All tests run in this JVM: [TestFieldCacheReopen, TestFieldCacheWithThreads, TestFieldCacheSortRandom, TestNumericTerms32, TestIndexSplitter, TestPKIndexSplitter, TestMultiPassIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest, TestLazyDocument, IndexSortingTest]
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSortingTest -Dtests.seed=4146977D8265D175 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=be_BY -Dtests.timezone=America/Los_Angeles -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>    [junit4] ERROR   0.00s J1 | IndexSortingTest (suite) <<<
>    [junit4]    > Throwable #1: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
>    [junit4]    >        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
>    [junit4]    >        at java.security.AccessController.checkPermission(AccessController.java:559)
>    [junit4]    >        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>    [junit4]    >        at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
>    [junit4]    >        at java.lang.Class.checkPackageAccess(Class.java:2309)
>    [junit4]    >        at java.lang.Class.checkMemberAccess(Class.java:2289)
>    [junit4]    >        at java.lang.Class.getDeclaredFields(Class.java:1810)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4] Completed [14/25] on J1 in 1.09s, 9 tests, 1 error <<< FAILURES!
>
> [...truncated 47 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 25 suites, 184 tests, 1 suite-level error, 1 ignored (1 assumption)
>
> Total time: 249 minutes 13 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
> Archived 4 artifacts
> Archive block size is 32768
> Received 0 blocks and 232124 bytes
> Compression is 0.0%
> Took 0.17 sec
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 953 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/953/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.index.IndexSortingTest

Error Message:
access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")

Stack Trace:
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
	at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
	at java.security.AccessController.checkPermission(AccessController.java:559)
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
	at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
	at java.lang.Class.checkPackageAccess(Class.java:2309)
	at java.lang.Class.checkMemberAccess(Class.java:2289)
	at java.lang.Class.getDeclaredFields(Class.java:1810)
	at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:573)
	at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSizeOf(RamUsageEstimator.java:537)
	at com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(RamUsageEstimator.java:385)
	at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:108)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 8052 lines...]
   [junit4] Suite: org.apache.lucene.index.IndexSortingTest
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {term_vectors=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=MockRandom), positions=FSTOrd50, docs=PostingsFormat(name=MockRandom), norm=PostingsFormat(name=Memory doPackFST= true)}, docValues:{sorted_set=DocValuesFormat(name=Memory), numeric=DocValuesFormat(name=Lucene50), binary=DocValuesFormat(name=Lucene50), sorted_numeric=DocValuesFormat(name=Asserting), sorted=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=be_BY, timezone=America/Los_Angeles
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=188195696,total=326107136
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldCacheReopen, TestFieldCacheWithThreads, TestFieldCacheSortRandom, TestNumericTerms32, TestIndexSplitter, TestPKIndexSplitter, TestMultiPassIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest, TestLazyDocument, IndexSortingTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSortingTest -Dtests.seed=4146977D8265D175 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=be_BY -Dtests.timezone=America/Los_Angeles -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J1 | IndexSortingTest (suite) <<<
   [junit4]    > Throwable #1: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.ch")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4146977D8265D175]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1525)
   [junit4]    > 	at java.lang.Class.checkPackageAccess(Class.java:2309)
   [junit4]    > 	at java.lang.Class.checkMemberAccess(Class.java:2289)
   [junit4]    > 	at java.lang.Class.getDeclaredFields(Class.java:1810)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [14/25] on J1 in 1.09s, 9 tests, 1 error <<< FAILURES!

[...truncated 47 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 25 suites, 184 tests, 1 suite-level error, 1 ignored (1 assumption)

Total time: 249 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 232124 bytes
Compression is 0.0%
Took 0.17 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 952 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/952/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:51047/q_n/f/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:51047/q_n/f/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([C43D50085530ADD4:4C696FD2FBCCC02C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#0): CloudJettyRunner [url=https://127.0.0.1:43745/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#0): CloudJettyRunner [url=https://127.0.0.1:43745/collection1]
	at __randomizedtesting.SeedInfo.seed([C43D50085530ADD4:4C696FD2FBCCC02C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=3699, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3699, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55971: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([C43D50085530ADD4]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=6888, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=6888, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51994/bc/ng: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([C43D50085530ADD4]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10306 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/init-core-data-001
   [junit4]   2> 174263 INFO  (SUITE-BasicDistributedZkTest-seed#[C43D50085530ADD4]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /q_n/f
   [junit4]   2> 174269 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 174283 INFO  (Thread-343) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 174283 INFO  (Thread-343) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 174382 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.ZkTestServer start zk server on port:54955
   [junit4]   2> 174383 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 174393 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 174402 INFO  (zkCallback-163-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@325ace80 name:ZooKeeperConnection Watcher:127.0.0.1:54955 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 174402 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 174403 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 174403 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 174406 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 174437 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 174439 INFO  (zkCallback-164-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7804a579 name:ZooKeeperConnection Watcher:127.0.0.1:54955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 174439 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 174439 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 174439 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 174442 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 174443 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 174444 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 174446 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 174446 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 174449 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 174449 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 174451 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 174451 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 174452 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 174453 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 174454 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 174455 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 174456 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 174456 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 174458 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 174458 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 174459 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 174460 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 174461 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 174461 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 174463 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 174463 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 174465 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 174465 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 174999 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1
   [junit4]   2> 175008 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 175061 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f594b0{/q_n/f,null,AVAILABLE}
   [junit4]   2> 175062 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@16f4b15b{HTTP/1.1}{127.0.0.1:40469}
   [junit4]   2> 175063 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.Server Started @178202ms
   [junit4]   2> 175063 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores, hostContext=/q_n/f, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/tempDir-001/control/data, hostPort=40469}
   [junit4]   2> 175070 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 175070 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/'
   [junit4]   2> 175086 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 175087 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 175093 INFO  (zkCallback-165-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@658957a7 name:ZooKeeperConnection Watcher:127.0.0.1:54955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 175093 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 175094 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 175101 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 175101 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/solr.xml
   [junit4]   2> 175129 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores
   [junit4]   2> 175135 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoreContainer New CoreContainer 1181736450
   [junit4]   2> 175135 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/]
   [junit4]   2> 175135 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/lib
   [junit4]   2> 175135 WARN  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/lib).
   [junit4]   2> 175156 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 175170 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 175174 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 175174 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 175174 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54955/solr
   [junit4]   2> 175174 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 175175 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 175198 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 175237 INFO  (zkCallback-167-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c84a0f6 name:ZooKeeperConnection Watcher:127.0.0.1:54955 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 175237 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 175238 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 175257 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 175261 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b038c4d name:ZooKeeperConnection Watcher:127.0.0.1:54955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 175261 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 175263 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 175265 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 175267 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 175269 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 175271 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 175273 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 175274 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 175275 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 175276 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 175287 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40469_q_n%2Ff
   [junit4]   2> 175287 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40469_q_n%2Ff
   [junit4]   2> 175289 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 175290 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 175291 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 175292 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000
   [junit4]   2> 175292 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40469_q_n%2Ff
   [junit4]   2> 175293 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 175294 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer Overseer (id=94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) starting
   [junit4]   2> 175296 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 175301 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 175313 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 175313 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 175325 INFO  (OverseerCollectionConfigSetProcessor-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 175329 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 175348 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 175348 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 175349 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores
   [junit4]   2> 175350 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/, transient=false, dataDir=data/, coreNodeName=, shard=, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1, loadOnStartup=true, config=solrconfig.xml, collection=control_collection, schema=schema.xml, name=collection1}
   [junit4]   2> 175351 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/
   [junit4]   2> 175351 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 175370 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 175370 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 175382 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 175386 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 175387 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 175387 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:40469_q_n%2Ff",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:40469/q_n/f",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 175388 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:40469_q_n%2Ff",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:40469/q_n/f",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 175388 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 175389 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 175391 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 175393 INFO  (zkCallback-168-thread-2-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 176382 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 176383 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 176383 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 176383 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 176384 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 176384 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/'
   [junit4]   2> 176426 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 176434 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 176474 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 176511 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 176513 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 176520 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 176952 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 176970 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 176972 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 176999 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 177003 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 177006 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 177007 WARN  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 177007 WARN  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 177007 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 177008 WARN  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 177008 WARN  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 177009 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 177009 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 177009 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 177010 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 177010 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25c7921e
   [junit4]   2> 177011 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/data
   [junit4]   2> 177011 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/data/index/
   [junit4]   2> 177011 WARN  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 177011 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/data/index
   [junit4]   2> 177012 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4516598280215486]
   [junit4]   2> 177013 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@4f3d856d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5002dfbf),segFN=segments_1,generation=1}
   [junit4]   2> 177014 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 177021 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 177022 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 177022 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 177022 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 177022 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 177023 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 177023 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 177023 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 177023 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 177026 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177028 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177029 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/data/
   [junit4]   2> 177029 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 177031 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177032 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177036 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,/admin/mbeans,/update/json/docs,/admin/luke,/admin/plugins,/schema,/admin/file,/replication,/get,/admin/system,/update,/admin/threads,standard,/admin/properties,/config,/admin/ping,/admin/logging,/update/csv,/admin/segments
   [junit4]   2> 177038 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 177056 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 177057 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 177058 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 177058 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 177058 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=9.0810546875, floorSegmentMB=0.2724609375, forceMergeDeletesPctAllowed=15.943449234028893, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 177059 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@4f3d856d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5002dfbf),segFN=segments_1,generation=1}
   [junit4]   2> 177059 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 177059 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2c0a00ae[collection1] main
   [junit4]   2> 177060 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 177061 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 177061 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 177062 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 177062 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 177062 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 177063 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 177063 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 177063 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 177063 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 177064 INFO  (searcherExecutor-380-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2c0a00ae[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 177064 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 177064 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 177064 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 177065 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511795044565123072
   [junit4]   2> 177067 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1511795044565123072
   [junit4]   2> 177067 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 177069 INFO  (coreLoadExecutor-379-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 177085 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:40469/q_n/f collection:control_collection shard:shard1
   [junit4]   2> 177086 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 177089 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94487190103654404-core_node1-n_0000000000
   [junit4]   2> 177090 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 177092 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 177092 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 177092 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40469/q_n/f/collection1/
   [junit4]   2> 177092 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 177092 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40469/q_n/f/collection1/ has no replicas
   [junit4]   2> 177093 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40469/q_n/f/collection1/ shard1
   [junit4]   2> 177093 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 177094 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 177095 INFO  (zkCallback-168-thread-2-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 177096 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 177099 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 177102 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40469/q_n/f",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 177206 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 177206 INFO  (zkCallback-168-thread-2-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 177254 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:40469/q_n/f/collection1/ and leader is http://127.0.0.1:40469/q_n/f/collection1/
   [junit4]   2> 177254 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40469/q_n/f
   [junit4]   2> 177254 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 177254 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 177254 INFO  (coreZkRegister-373-thread-1-processing-r:core_node1 n:127.0.0.1:40469_q_n%2Ff c:control_collection x:collection1 s:shard1) [n:127.0.0.1:40469_q_n%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 177256 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:40469_q_n%2Ff",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:40469/q_n/f",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1"} current state version: 3
   [junit4]   2> 177257 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:40469_q_n%2Ff",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:40469/q_n/f",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 177360 INFO  (zkCallback-168-thread-2-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 177360 INFO  (zkCallback-168-thread-1-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 177396 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177446 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177462 INFO  (zkCallback-170-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@429800ef name:ZooKeeperConnection Watcher:127.0.0.1:54955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177462 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177462 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177462 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 177465 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 177465 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 177466 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177470 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177470 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d3c1396 name:ZooKeeperConnection Watcher:127.0.0.1:54955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177471 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177471 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177474 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 177474 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 177577 INFO  (OverseerStateUpdate-94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 177836 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/cores/collection1
   [junit4]   2> 177837 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001
   [junit4]   2> 177838 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 177842 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f7ee9e4{/q_n/f,null,AVAILABLE}
   [junit4]   2> 177843 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@30346b90{HTTP/1.1}{127.0.0.1:51047}
   [junit4]   2> 177843 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.e.j.s.Server Started @180982ms
   [junit4]   2> 177843 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/q_n/f, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/tempDir-001/jetty1, hostPort=51047, solrconfig=solrconfig.xml}
   [junit4]   2> 177844 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 177844 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/'
   [junit4]   2> 177864 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177870 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177887 INFO  (zkCallback-172-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3599b255 name:ZooKeeperConnection Watcher:127.0.0.1:54955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177887 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177889 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177893 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 177893 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/solr.xml
   [junit4]   2> 177931 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/cores
   [junit4]   2> 177931 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoreContainer New CoreContainer 953641045
   [junit4]   2> 177932 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/]
   [junit4]   2> 177932 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/lib
   [junit4]   2> 177932 WARN  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/lib).
   [junit4]   2> 177952 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 177958 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 177959 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 177959 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 177960 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54955/solr
   [junit4]   2> 177960 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 177961 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177965 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177970 INFO  (zkCallback-174-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26157f4a name:ZooKeeperConnection Watcher:127.0.0.1:54955 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177971 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177973 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177993 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177995 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:51047_q_n%2Ff) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f67bcdd name:ZooKeeperConnection Watcher:127.0.0.1:54955/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177995 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 178004 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 179013 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51047_q_n%2Ff
   [junit4]   2> 179014 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51047_q_n%2Ff
   [junit4]   2> 179015 INFO  (zkCallback-170-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 179015 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:51047_q_n%2Ff) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 179016 INFO  (zkCallback-168-thread-2-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 179016 INFO  (zkCallback-168-thread-3-processing-n:127.0.0.1:40469_q_n%2Ff) [n:127.0.0.1:40469_q_n%2Ff    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 179017 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 179018 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94487190103654409-127.0.0.1:51047_q_n%2Ff-n_0000000001
   [junit4]   2> 179019 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94487190103654404-127.0.0.1:40469_q_n%2Ff-n_0000000000 to know if I could be the leader
   [junit4]   2> 179044 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 179044 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 179045 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/cores
   [junit4]   2> 179046 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, name=collection1, configSetProperties=configsetprops.json, loadOnStartup=true, config=solrconfig.xml, transient=false, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/cores/collection1/, collection=collection1, shard=, coreNodeName=}
   [junit4]   2> 179046 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_C43D50085530ADD4-001/shard-1-001/cores/collection1/
   [junit4]   2> 179046 INFO  (TEST-BasicDistributedZkTest.test-seed#[C43D50085530ADD4]) [n:127.0.0.1:51047_q_n%2Ff    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 179048 INFO  (coreLoadExecutor-390-thread-1-processing-n:127.0.0.1:51047_q_n%2Ff) [n:127.0.0.1:51047_q_n%2Ff c:collection1   x:collection1]

[...truncated too long message...]

ocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor89.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 1931952 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C43D50085530ADD4]-worker) [n:127.0.0.1:51994_bc%2Fng c:addReplicaColl s:shard1 r:core_node2 x:addReplicaColl_shard1_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 08, 2015 9:24:20 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=fr_CH, timezone=America/Virgin
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=6,free=116987704,total=529006592
   [junit4]   2> NOTE: All tests run in this JVM: [TestTrackingShardHandlerFactory, TestSchemaNameResource, TestPerFieldSimilarity, TestFuzzyAnalyzedSuggestions, DistributedExpandComponentTest, TestExtendedDismaxParser, TestStressRecovery, TestGroupingSearch, HardAutoCommitTest, TestChildDocTransformer, TestDefaultSimilarityFactory, BinaryUpdateRequestHandlerTest, SuggesterTSTTest, TestCharFilters, AliasIntegrationTest, TestSort, StatsComponentTest, TestConfigSetImmutable, AutoCommitTest, UnloadDistributedZkTest, RemoteQueryErrorTest, DateRangeFieldTest, RecoveryAfterSoftCommitTest, TestConfigSetProperties, FieldAnalysisRequestHandlerTest, TestFileDictionaryLookup, TestRemoveLastDynamicCopyField, RuleEngineTest, PrimitiveFieldTypeTest, SimpleFacetsTest, DocValuesMultiTest, CollectionTooManyReplicasTest, UpdateParamsTest, TestSimpleTrackingShardHandler, TestSchemaManager, AddSchemaFieldsUpdateProcessorFactoryTest, OverseerStatusTest, TestScoreJoinQPScore, AssignTest, TestSolrXml, AsyncMigrateRouteKeyTest, SearchHandlerTest, SuggesterWFSTTest, TestCloudSchemaless, BadIndexSchemaTest, CurrencyFieldXmlFileTest, FullSolrCloudDistribCmdsTest, CustomCollectionTest, HighlighterConfigTest, RollingRestartTest, TestRuleBasedAuthorizationPlugin, SystemInfoHandlerTest, TestCloudPivotFacet, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [208/537] on J2 in 713.52s, 1 test, 1 error <<< FAILURES!

[...truncated 1048 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 537 suites, 2124 tests, 2 errors, 2 failures, 29 ignored (17 assumptions)

Total time: 198 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1215458862 bytes
Compression is 0.0%
Took 3 min 20 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 951 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/951/

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
ERROR: SolrIndexSearcher opens=17 closes=16

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=17 closes=16
	at __randomizedtesting.SeedInfo.seed([39CE6211379E0B58]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:467)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:233)
	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:     1) Thread[id=6187, name=qtp55414109-6187, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.blockUntilFinished(ConcurrentUpdateSolrClient.java:404)         at org.apache.solr.update.StreamingSolrClients.blockUntilFinished(StreamingSolrClients.java:103)         at org.apache.solr.update.SolrCmdDistributor.blockAndDoRetries(SolrCmdDistributor.java:231)         at org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:89)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:781)         at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1648)         at org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:183)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:667)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=6207, name=searcherExecutor-2263-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
   1) Thread[id=6187, name=qtp55414109-6187, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:503)
        at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.blockUntilFinished(ConcurrentUpdateSolrClient.java:404)
        at org.apache.solr.update.StreamingSolrClients.blockUntilFinished(StreamingSolrClients.java:103)
        at org.apache.solr.update.SolrCmdDistributor.blockAndDoRetries(SolrCmdDistributor.java:231)
        at org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:89)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:781)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1648)
        at org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:183)
        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
        at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:667)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=6207, name=searcherExecutor-2263-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([39CE6211379E0B58]:0)


REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:59479/_g/nj: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59479/_g/nj: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1441668844055,generation=2,filelist=[_a9.fdt, _a9.fdx, _a9.fnm, _a9.nvd, _a9.nvm, _a9.si, _a9_Memory_0.ram, _ad.fdt, _ad.fdx, _ad.fnm, _ad.nvd, _ad.nvm, _ad.si, _ad_Memory_0.ram, _ae.cfe, _ae.cfs, _ae.si, _ah.fdt, _ah.fdx, _ah.fnm, _ah.nvd, _ah.nvm, _ah.si, _ah_Memory_0.ram, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, _al.fdt, _al.fdx, _al.fnm, _al.nvd, _al.nvm, _al.si, _al_Memory_0.ram, _am.fdt, _am.fdx, _am.fnm, _am.nvd, _am.nvm, _am.si, _am_Memory_0.ram, _an.fdt, _an.fdx, _an.fnm, _an.nvd, _an.nvm, _an.si, _an_Memory_0.ram, _ap.fdt, _ap.fdx, _ap.fnm, _ap.nvd, _ap.nvm, _ap.si, _ap_Memory_0.ram, _aq.fdt, _aq.fdx, _aq.fnm, _aq.nvd, _aq.nvm, _aq.si, _aq_Memory_0.ram, segments_2]}]> but was:<[{indexVersion=1441668844055,generation=2,filelist=[_a9.fdt, _a9.fdx, _a9.fnm, _a9.nvd, _a9.nvm, _a9.si, _a9_Memory_0.ram, _ad.fdt, _ad.fdx, _ad.fnm, _ad.nvd, _ad.nvm, _ad.si, _ad_Memory_0.ram, _ae.cfe, _ae.cfs, _ae.si, _ah.fdt, _ah.fdx, _ah.fnm, _ah.nvd, _ah.nvm, _ah.si, _ah_Memory_0.ram, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, _al.fdt, _al.fdx, _al.fnm, _al.nvd, _al.nvm, _al.si, _al_Memory_0.ram, _am.fdt, _am.fdx, _am.fnm, _am.nvd, _am.nvm, _am.si, _am_Memory_0.ram, _an.fdt, _an.fdx, _an.fnm, _an.nvd, _an.nvm, _an.si, _an_Memory_0.ram, _ap.fdt, _ap.fdx, _ap.fnm, _ap.nvd, _ap.nvm, _ap.si, _ap_Memory_0.ram, _aq.fdt, _aq.fdx, _aq.fnm, _aq.nvd, _aq.nvm, _aq.si, _aq_Memory_0.ram, segments_2]}, {indexVersion=1441668844055,generation=3,filelist=[_an.fdt, _an.fdx, _an.fnm, _an.nvd, _an.nvm, _an.si, _an_Memory_0.ram, _ao.cfe, _ao.cfs, _ao.si, _ap.fdt, _ap.fdx, _ap.fnm, _ap.nvd, _ap.nvm, _ap.si, _ap_Memory_0.ram, _aq.fdt, _aq.fdx, _aq.fnm, _aq.nvd, _aq.nvm, _aq.si, _aq_Memory_0.ram, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1441668844055,generation=2,filelist=[_a9.fdt, _a9.fdx, _a9.fnm, _a9.nvd, _a9.nvm, _a9.si, _a9_Memory_0.ram, _ad.fdt, _ad.fdx, _ad.fnm, _ad.nvd, _ad.nvm, _ad.si, _ad_Memory_0.ram, _ae.cfe, _ae.cfs, _ae.si, _ah.fdt, _ah.fdx, _ah.fnm, _ah.nvd, _ah.nvm, _ah.si, _ah_Memory_0.ram, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, _al.fdt, _al.fdx, _al.fnm, _al.nvd, _al.nvm, _al.si, _al_Memory_0.ram, _am.fdt, _am.fdx, _am.fnm, _am.nvd, _am.nvm, _am.si, _am_Memory_0.ram, _an.fdt, _an.fdx, _an.fnm, _an.nvd, _an.nvm, _an.si, _an_Memory_0.ram, _ap.fdt, _ap.fdx, _ap.fnm, _ap.nvd, _ap.nvm, _ap.si, _ap_Memory_0.ram, _aq.fdt, _aq.fdx, _aq.fnm, _aq.nvd, _aq.nvm, _aq.si, _aq_Memory_0.ram, segments_2]}]> but was:<[{indexVersion=1441668844055,generation=2,filelist=[_a9.fdt, _a9.fdx, _a9.fnm, _a9.nvd, _a9.nvm, _a9.si, _a9_Memory_0.ram, _ad.fdt, _ad.fdx, _ad.fnm, _ad.nvd, _ad.nvm, _ad.si, _ad_Memory_0.ram, _ae.cfe, _ae.cfs, _ae.si, _ah.fdt, _ah.fdx, _ah.fnm, _ah.nvd, _ah.nvm, _ah.si, _ah_Memory_0.ram, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, _al.fdt, _al.fdx, _al.fnm, _al.nvd, _al.nvm, _al.si, _al_Memory_0.ram, _am.fdt, _am.fdx, _am.fnm, _am.nvd, _am.nvm, _am.si, _am_Memory_0.ram, _an.fdt, _an.fdx, _an.fnm, _an.nvd, _an.nvm, _an.si, _an_Memory_0.ram, _ap.fdt, _ap.fdx, _ap.fnm, _ap.nvd, _ap.nvm, _ap.si, _ap_Memory_0.ram, _aq.fdt, _aq.fdx, _aq.fnm, _aq.nvd, _aq.nvm, _aq.si, _aq_Memory_0.ram, segments_2]}, {indexVersion=1441668844055,generation=3,filelist=[_an.fdt, _an.fdx, _an.fnm, _an.nvd, _an.nvm, _an.si, _an_Memory_0.ram, _ao.cfe, _ao.cfs, _ao.si, _ap.fdt, _ap.fdx, _ap.fnm, _ap.nvd, _ap.nvm, _ap.si, _ap_Memory_0.ram, _aq.fdt, _aq.fdx, _aq.fnm, _aq.nvd, _aq.nvm, _aq.si, _aq_Memory_0.ram, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([39CE6211379E0B58:1C19792147D6055B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=39229, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=39229, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53929: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([39CE6211379E0B58]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10666 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/init-core-data-001
   [junit4]   2> 1208552 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[39CE6211379E0B58]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1208557 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1208558 INFO  (Thread-3627) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1208558 INFO  (Thread-3627) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1208658 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.ZkTestServer start zk server on port:56061
   [junit4]   2> 1208658 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1208660 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1208663 INFO  (zkCallback-419-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@751144b3 name:ZooKeeperConnection Watcher:127.0.0.1:56061 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1208663 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1208664 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1208664 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1208667 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1208668 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1208669 INFO  (zkCallback-420-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a93e0d7 name:ZooKeeperConnection Watcher:127.0.0.1:56061/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1208670 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1208670 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1208670 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1208673 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1208674 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1208676 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1208680 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1208680 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1208684 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1208685 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1208690 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1208690 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1208693 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1208693 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1208696 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1208697 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1208699 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1208700 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1208703 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1208703 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1208713 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1208713 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1208716 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1208716 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1208720 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1208721 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1208723 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1208723 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1209010 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1
   [junit4]   2> 1209012 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1209021 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@80280e2{/,null,AVAILABLE}
   [junit4]   2> 1209031 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.ServerConnector Started ServerConnector@798840bd{HTTP/1.1}{127.0.0.1:59298}
   [junit4]   2> 1209031 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.Server Started @1212004ms
   [junit4]   2> 1209031 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores, hostContext=/, hostPort=59298, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/tempDir-001/control/data}
   [junit4]   2> 1209032 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 1209032 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/'
   [junit4]   2> 1209061 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1209073 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1209075 INFO  (zkCallback-421-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ec156fd name:ZooKeeperConnection Watcher:127.0.0.1:56061/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1209076 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1209076 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1209078 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1209078 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/solr.xml
   [junit4]   2> 1209097 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores
   [junit4]   2> 1209098 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoreContainer New CoreContainer 1415108305
   [junit4]   2> 1209098 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/]
   [junit4]   2> 1209098 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/lib
   [junit4]   2> 1209098 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/lib).
   [junit4]   2> 1209112 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1209115 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1209116 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1209116 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1209116 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56061/solr
   [junit4]   2> 1209116 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1209116 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1209120 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1209122 INFO  (zkCallback-423-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74baca5b name:ZooKeeperConnection Watcher:127.0.0.1:56061 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1209122 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1209122 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1209168 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1209186 INFO  (zkCallback-424-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a6f9ff9 name:ZooKeeperConnection Watcher:127.0.0.1:56061/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1209186 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1209187 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1209189 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1209192 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1209193 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1209195 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1209197 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1209198 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1209199 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1209200 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1209201 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59298_
   [junit4]   2> 1209201 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59298_
   [junit4]   2> 1209203 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1209204 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1209205 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1209205 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94481134461648900-127.0.0.1:59298_-n_0000000000
   [junit4]   2> 1209206 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59298_
   [junit4]   2> 1209206 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1209207 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.Overseer Overseer (id=94481134461648900-127.0.0.1:59298_-n_0000000000) starting
   [junit4]   2> 1209208 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1209222 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1209227 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1209227 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1209229 INFO  (OverseerCollectionConfigSetProcessor-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1209257 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1209276 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1209276 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1209287 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores
   [junit4]   2> 1209291 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/, coreNodeName=, name=collection1, collection=control_collection, dataDir=data/, config=solrconfig.xml, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1, configSetProperties=configsetprops.json, transient=false, loadOnStartup=true}
   [junit4]   2> 1209292 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/
   [junit4]   2> 1209293 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1209299 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1209299 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1209313 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1209313 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [n:127.0.0.1:59298_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1209313 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:59298_",
   [junit4]   2>   "base_url":"http://127.0.0.1:59298",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1209314 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1209314 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:59298_",
   [junit4]   2>   "base_url":"http://127.0.0.1:59298",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1209315 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1209315 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1209317 INFO  (zkCallback-424-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1209332 INFO  (zkCallback-424-thread-2-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1210315 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1210315 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1210316 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1210316 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1210316 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1210316 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/'
   [junit4]   2> 1210339 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1210357 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1210372 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1210423 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1210424 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1210441 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1210802 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1210804 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1210805 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1210810 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1210840 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1210840 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1210841 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 1210841 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1210841 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46c22635
   [junit4]   2> 1210842 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/data
   [junit4]   2> 1210842 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1210842 WARN  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1210843 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/data/index
   [junit4]   2> 1210843 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=28, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.45478845448675276]
   [junit4]   2> 1210859 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 1210859 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1210870 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/data/
   [junit4]   2> 1210872 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1210872 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1210873 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1210873 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1210873 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1210873 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1210874 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1210874 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1210874 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1210885 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1210890 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1210894 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1210899 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1210911 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/segments,/admin/logging,/admin/file,/update/json/docs,/get,/update/json,/replication,/admin/ping,/admin/luke,/admin/plugins,/update,standard,/admin/properties,/admin/mbeans,/admin/system,/schema,/config,/update/csv
   [junit4]   2> 1210914 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1210915 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1210915 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1210918 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1210918 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1210919 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=28, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1210920 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 1210921 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1210922 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2660d44f[collection1] main
   [junit4]   2> 1210922 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1210923 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1210923 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1210924 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1210924 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1210925 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1210925 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1210925 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1210926 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1210926 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1210927 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1210927 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1210927 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1210927 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511698153868361728
   [junit4]   2> 1210927 INFO  (searcherExecutor-2252-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2660d44f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1210931 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 4.0ms to seed version buckets with highest version 1511698153868361728
   [junit4]   2> 1210932 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1210932 INFO  (coreLoadExecutor-2251-thread-1-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1210934 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59298 collection:control_collection shard:shard1
   [junit4]   2> 1210935 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1210943 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94481134461648900-core_node1-n_0000000000
   [junit4]   2> 1210943 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1210945 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1210946 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1210946 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59298/collection1/
   [junit4]   2> 1210946 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1210946 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59298/collection1/ has no replicas
   [junit4]   2> 1210946 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59298/collection1/ shard1
   [junit4]   2> 1210947 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1210951 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1210952 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1210959 INFO  (zkCallback-424-thread-3-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1210959 INFO  (zkCallback-424-thread-2-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1210983 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59298",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1211086 INFO  (zkCallback-424-thread-2-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211086 INFO  (zkCallback-424-thread-3-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211136 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:59298/collection1/ and leader is http://127.0.0.1:59298/collection1/
   [junit4]   2> 1211136 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59298
   [junit4]   2> 1211136 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1211136 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1211137 INFO  (coreZkRegister-2245-thread-1-processing-n:127.0.0.1:59298_ x:collection1 r:core_node1 c:control_collection s:shard1) [n:127.0.0.1:59298_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1211140 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:59298_",
   [junit4]   2>   "base_url":"http://127.0.0.1:59298",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 1211141 INFO  (OverseerStateUpdate-94481134461648900-127.0.0.1:59298_-n_0000000000) [n:127.0.0.1:59298_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:59298_",
   [junit4]   2>   "base_url":"http://127.0.0.1:59298",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1211244 INFO  (zkCallback-424-thread-2-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211244 INFO  (zkCallback-424-thread-3-processing-n:127.0.0.1:59298_) [n:127.0.0.1:59298_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211315 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1211324 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1211333 INFO  (zkCallback-426-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66269f68 name:ZooKeeperConnection Watcher:127.0.0.1:56061/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1211333 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1211334 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1211334 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1211337 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1211757 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/cores/collection1
   [junit4]   2> 1211758 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001
   [junit4]   2> 1211759 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1211762 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@715f5a08{/,null,AVAILABLE}
   [junit4]   2> 1211762 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2b0ffd5e{HTTP/1.1}{127.0.0.1:34359}
   [junit4]   2> 1211762 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.e.j.s.Server Started @1214735ms
   [junit4]   2> 1211762 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34359, hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/cores}
   [junit4]   2> 1211763 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 1211763 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/'
   [junit4]   2> 1211796 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1211800 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1211801 INFO  (zkCallback-427-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@439390d7 name:ZooKeeperConnection Watcher:127.0.0.1:56061/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1211801 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1211801 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1211803 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1211803 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/solr.xml
   [junit4]   2> 1211824 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/cores
   [junit4]   2> 1211824 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoreContainer New CoreContainer 1060055264
   [junit4]   2> 1211825 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/]
   [junit4]   2> 1211825 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/lib
   [junit4]   2> 1211825 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_39CE6211379E0B58-001/shard-1-001/lib).
   [junit4]   2> 1211838 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1211841 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1211843 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1211844 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE6211379E0B58]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1211844 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[39CE62

[...truncated too long message...]

omizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 5581831 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[39CE6211379E0B58]-worker) [n:127.0.0.1:53929_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 07, 2015 8:27:55 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {intDefault=Lucene50(blocksize=128), other_tl1=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=FSTOrd50, a_i1=FSTOrd50, range_facet_l=FSTOrd50, text=FST50, timestamp=FSTOrd50, a_t=FSTOrd50, _version_=Lucene50(blocksize=128), range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=FSTOrd50}, docValues:{range_facet_i_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=SimpleText)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=es_EC, timezone=America/Nome
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=152091536,total=530055168
   [junit4]   2> NOTE: All tests run in this JVM: [SimpleMLTQParserTest, TestJmxMonitoredMap, SpatialRPTFieldTypeTest, LeaderFailoverAfterPartitionTest, RequestLoggingTest, TestWordDelimiterFilterFactory, CSVRequestHandlerTest, SchemaVersionSpecificBehaviorTest, AnalysisErrorHandlingTest, TestExactStatsCache, TestMacros, TestManagedResourceStorage, TestFieldTypeResource, TestClassNameShortening, BasicFunctionalityTest, FastVectorHighlighterTest, TestJsonFacets, TestDocBasedVersionConstraints, TestDownShardTolerantSearch, TestBinaryResponseWriter, TestRebalanceLeaders, ChaosMonkeySafeLeaderTest, TestSuggestSpellingConverter, MultiThreadedOCPTest, SliceStateTest, AsyncCallRequestStatusResponseTest, SaslZkACLProviderTest, TestCodecSupport, TestDynamicFieldCollectionResource, TestSchemaSimilarityResource, MoreLikeThisHandlerTest, TestDistributedGrouping, CollectionStateFormat2Test, ZkCLITest, DataDrivenBlockJoinTest, NotRequiredUniqueKeyTest, SuggesterFSTTest, SolrTestCaseJ4Test, ConvertedLegacyTest, SpatialHeatmapFacetsTest, TestDynamicFieldResource, BJQParserTest, DistributedFacetPivotSmallTest, TestSolrDynamicMBean, UniqFieldsUpdateProcessorFactoryTest, MBeansHandlerTest, PreAnalyzedFieldTest, OpenCloseCoreStressTest, TestQueryUtils, UpdateRequestProcessorFactoryTest, SpellCheckComponentTest, TestUpdate, MinimalSchemaTest, DirectSolrSpellCheckerTest, DistributedTermsComponentTest, TestReplicationHandlerBackup, TestSolrQueryParserDefaultOperatorResource, DOMUtilTest, ChaosMonkeyNothingIsSafeTest, ClusterStateUpdateTest, MultiTermTest, DistanceUnitsTest, TestCloudInspectUtil, TestExpandComponent, TestPartialUpdateDeduplication, DistributedDebugComponentTest, TermsComponentTest, TestNonDefinedSimilarityFactory, TestInitQParser, SolrIndexConfigTest, TestDistribIDF, HdfsNNFailoverTest, TestSerializedLuceneMatchVersion, HighlighterMaxOffsetTest, OutOfBoxZkACLAndCredentialsProvidersTest, TermVectorComponentTest, BasicAuthIntegrationTest, TestManagedStopFilterFactory, TestSolrConfigHandlerCloud, SpatialFilterTest, TestMiniSolrCloudClusterBase, TestLMJelinekMercerSimilarityFactory, TestSolrCloudWithKerberosAlt, TestJettySolrRunner, CircularListTest, WordBreakSolrSpellCheckerTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, ShardRoutingCustomTest, AutoCommitTest, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestReload, TestStressVersions, HardAutoCommitTest, TestCoreContainer, TestSolr4Spatial, TestGroupingSearch, SolrCmdDistributorTest, PeerSyncTest, TestFiltering, DirectUpdateHandlerTest, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, CoreAdminHandlerTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTSTTest, TestStressLucene, TestTrie, PolyFieldTest, NoCacheHeaderTest, TestAtomicUpdateErrorCases, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestWriterPerf, TestQueryTypes, PrimitiveFieldTypeTest, RequestHandlersTest, DocumentBuilderTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, SolrPluginUtilsTest, TestJmxIntegration, JSONWriterTest, QueryParsingTest, BinaryUpdateRequestHandlerTest, TestPHPSerializedResponseWriter, TestLFUCache, HighlighterConfigTest, SOLR749Test, TestQuerySenderListener, AlternateDirectoryTest, TestQuerySenderNoQuery, ResponseLogComponentTest, TestStressRecovery, NumericFieldsTest, TestPhraseSuggestions, SynonymTokenizerTest, EchoParamsTest, TestDFRSimilarityFactory, TestBM25SimilarityFactory, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, DateMathParserTest, PrimUtilsTest, TestSolrJ, TestUtils, UUIDFieldTest, TestCrossCoreJoin, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, AssignTest, CleanupOldIndexTest, CollectionsAPIAsyncDistributedZkTest, ConcurrentDeleteAndCreateCollectionTest, DeleteReplicaTest, DistribJoinFromCollectionTest, DistributedQueueTest, DistributedVersionInfoTest, SharedFSAutoReplicaFailoverUtilsTest, SolrXmlInZkTest, TestAuthenticationFramework, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestCryptoKeys, TestMiniSolrCloudCluster, TestRandomRequestDistribution, TestRequestStatusCollectionAPI, TestShortCircuitedRequests, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [537/537] on J1 in 2042.74s, 1 test, 1 error <<< FAILURES!

[...truncated 14 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 537 suites, 2124 tests, 2 suite-level errors, 2 errors, 1 failure, 29 ignored (17 assumptions)

Total time: 216 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1292078992 bytes
Compression is 0.0%
Took 3 min 38 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 950 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/950/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=16260, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16260, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43079: Could not find collection : awholynewstresscollection_collection2_1
	at __randomizedtesting.SeedInfo.seed([462C4EE3F1F34FFF]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10831 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/init-core-data-001
   [junit4]   2> 1873656 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1873656 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1873787 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1873813 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1873817 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1873841 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_46563_hdfs____cosd5m/webapp
   [junit4]   2> 1874078 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1874623 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46563
   [junit4]   2> 1874857 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1874859 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1874874 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_60104_datanode____.7ltd3y/webapp
   [junit4]   2> 1875012 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1875464 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60104
   [junit4]   2> 1875539 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1875541 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1875576 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41937_datanode____rz2sh3/webapp
   [junit4]   2> 1875673 INFO  (IPC Server handler 3 on 56897) [    ] BlockStateChange BLOCK* processReport: from storage DS-1c7eb8e8-0de4-459b-9d32-6cbb85cbad69 node DatanodeRegistration(127.0.0.1, datanodeUuid=e36b0407-aa87-4c4b-9689-bd0b2134edad, infoPort=60104, ipcPort=41471, storageInfo=lv=-56;cid=testClusterID;nsid=1710479821;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1875674 INFO  (IPC Server handler 3 on 56897) [    ] BlockStateChange BLOCK* processReport: from storage DS-70f0d6c4-3b4d-481a-a7eb-654075270e44 node DatanodeRegistration(127.0.0.1, datanodeUuid=e36b0407-aa87-4c4b-9689-bd0b2134edad, infoPort=60104, ipcPort=41471, storageInfo=lv=-56;cid=testClusterID;nsid=1710479821;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1875765 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1876266 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41937
   [junit4]   2> 1876425 INFO  (IPC Server handler 5 on 56897) [    ] BlockStateChange BLOCK* processReport: from storage DS-e0197b79-3047-47b0-a0bc-ca995870a27d node DatanodeRegistration(127.0.0.1, datanodeUuid=56c964ac-7f6b-4975-9951-fa130fb2c767, infoPort=41937, ipcPort=54887, storageInfo=lv=-56;cid=testClusterID;nsid=1710479821;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 1876426 INFO  (IPC Server handler 5 on 56897) [    ] BlockStateChange BLOCK* processReport: from storage DS-aca98d5f-fe77-425a-a2b3-34d2b93edc6e node DatanodeRegistration(127.0.0.1, datanodeUuid=56c964ac-7f6b-4975-9951-fa130fb2c767, infoPort=41937, ipcPort=54887, storageInfo=lv=-56;cid=testClusterID;nsid=1710479821;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1876556 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1876557 INFO  (Thread-9469) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1876557 INFO  (Thread-9469) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1876658 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.ZkTestServer start zk server on port:37503
   [junit4]   2> 1876659 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1876663 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1876665 INFO  (zkCallback-1314-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@161a0a46 name:ZooKeeperConnection Watcher:127.0.0.1:37503 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1876666 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1876667 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1876667 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1876671 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1876672 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1876674 INFO  (zkCallback-1315-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5130f40a name:ZooKeeperConnection Watcher:127.0.0.1:37503/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1876674 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1876674 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1876674 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1876677 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1876679 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1876681 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1876683 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1876683 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1876687 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1876687 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1876691 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1876691 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1876694 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1876694 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1876697 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1876697 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1876699 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1876700 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1876702 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1876702 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1876706 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1876706 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1876709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1876709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1876721 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1876722 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1876726 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1876726 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1877034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores/collection1
   [junit4]   2> 1877036 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1877039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b988695{/,null,AVAILABLE}
   [junit4]   2> 1877040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.ServerConnector Started ServerConnector@25aed246{HTTP/1.1}{127.0.0.1:59386}
   [junit4]   2> 1877040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.Server Started @1880386ms
   [junit4]   2> 1877040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=59386, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores, hostContext=/, solr.data.dir=hdfs://localhost:56897/hdfs__localhost_56897__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001_tempDir-002_control_data}
   [junit4]   2> 1877041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1877041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/'
   [junit4]   2> 1877060 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1877061 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1877062 INFO  (zkCallback-1316-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@430fcf54 name:ZooKeeperConnection Watcher:127.0.0.1:37503/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1877062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1877062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1877063 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1877064 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/solr.xml
   [junit4]   2> 1877080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores
   [junit4]   2> 1877081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoreContainer New CoreContainer 811386040
   [junit4]   2> 1877081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/]
   [junit4]   2> 1877081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/lib
   [junit4]   2> 1877081 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/lib).
   [junit4]   2> 1877090 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1877091 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1877092 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1877092 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1877092 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37503/solr
   [junit4]   2> 1877092 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1877093 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1877094 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1877094 INFO  (zkCallback-1318-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f5bf05c name:ZooKeeperConnection Watcher:127.0.0.1:37503 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1877096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1877096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1877098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1877099 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50727999 name:ZooKeeperConnection Watcher:127.0.0.1:37503/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1877100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1877101 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1877103 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1877109 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1877112 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1877116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1877120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1877123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1877124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1877127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1877130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59386_
   [junit4]   2> 1877130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59386_
   [junit4]   2> 1877132 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1877133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1877137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1877137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94475563270995972-127.0.0.1:59386_-n_0000000000
   [junit4]   2> 1877138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59386_
   [junit4]   2> 1877138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1877140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer Overseer (id=94475563270995972-127.0.0.1:59386_-n_0000000000) starting
   [junit4]   2> 1877142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1877152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1877153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1877197 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1877201 INFO  (OverseerCollectionConfigSetProcessor-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1877217 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1877218 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1877219 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores
   [junit4]   2> 1877220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, config=solrconfig.xml, configSetProperties=configsetprops.json, shard=, transient=false, loadOnStartup=true, dataDir=data/, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores/collection1, name=collection1, collection=control_collection}
   [junit4]   2> 1877220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores/collection1/
   [junit4]   2> 1877220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1877222 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1877222 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1877222 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1877223 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:59386_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1877224 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:59386_",
   [junit4]   2>   "base_url":"http://127.0.0.1:59386"} current state version: 0
   [junit4]   2> 1877225 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:59386_",
   [junit4]   2>   "base_url":"http://127.0.0.1:59386"}
   [junit4]   2> 1877225 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1877226 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1877226 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1877228 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1878226 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1878226 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1878227 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1878227 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1878228 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1878228 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores/collection1/'
   [junit4]   2> 1878248 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1878261 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1878296 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1878405 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1878406 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1878424 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1878807 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1878829 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1878843 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1878859 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1878867 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1878871 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1878871 WARN  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1878872 WARN  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1878872 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1878872 WARN  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1878873 WARN  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1878874 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1878875 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1878875 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1878876 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56897/solr_hdfs_home
   [junit4]   2> 1878876 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1878876 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1878877 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3376b54c
   [junit4]   2> 1878877 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1878904 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 1878904 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1878905 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1878918 WARN  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:56897/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1878920 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1878960 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 1878961 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1878962 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=93.8212890625, floorSegmentMB=0.8232421875, forceMergeDeletesPctAllowed=9.151995794582025, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.296344098460823
   [junit4]   2> 1878994 INFO  (IPC Server handler 3 on 56897) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57541 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-e0197b79-3047-47b0-a0bc-ca995870a27d:NORMAL:127.0.0.1:53972|RBW], ReplicaUnderConstruction[[DISK]DS-1c7eb8e8-0de4-459b-9d32-6cbb85cbad69:NORMAL:127.0.0.1:57541|RBW]]} size 0
   [junit4]   2> 1878996 INFO  (IPC Server handler 4 on 56897) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:53972 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-e0197b79-3047-47b0-a0bc-ca995870a27d:NORMAL:127.0.0.1:53972|RBW], ReplicaUnderConstruction[[DISK]DS-1c7eb8e8-0de4-459b-9d32-6cbb85cbad69:NORMAL:127.0.0.1:57541|RBW]]} size 0
   [junit4]   2> 1879002 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8c56553 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@47b5f37a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1879002 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1879003 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:56897/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1879008 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1879008 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1879009 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1879009 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1879009 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1879009 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1879010 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1879010 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1879010 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1879014 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1879016 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1879020 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1879022 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1879026 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/admin/plugins,/admin/file,/update/json/docs,/admin/mbeans,/replication,/admin/ping,/update/json,/admin/luke,/admin/properties,/update/csv,/admin/logging,standard,/get,/admin/threads,/update,/config,/admin/system,/schema
   [junit4]   2> 1879028 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1879029 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1879029 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 1879048 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1879048 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1879049 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=49.0625, floorSegmentMB=0.701171875, forceMergeDeletesPctAllowed=29.090467671983582, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1879060 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8c56553 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@47b5f37a; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1879061 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1879061 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@e81ccb7[collection1] main
   [junit4]   2> 1879061 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1879063 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1879064 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1879064 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1879064 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1879064 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1879065 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1879065 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1879065 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1879066 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1879067 INFO  (searcherExecutor-3786-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@e81ccb7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1879071 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1879071 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 1879072 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1879072 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511609014965764096
   [junit4]   2> 1879072 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1511609014965764096
   [junit4]   2> 1879073 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1879073 INFO  (coreLoadExecutor-3785-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1879074 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59386 collection:control_collection shard:shard1
   [junit4]   2> 1879074 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1879082 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94475563270995972-core_node1-n_0000000000
   [junit4]   2> 1879083 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1879088 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1879089 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1879089 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1879089 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59386/collection1/
   [junit4]   2> 1879089 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1879089 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59386/collection1/ has no replicas
   [junit4]   2> 1879089 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59386/collection1/ shard1
   [junit4]   2> 1879092 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1879093 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1879101 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1879105 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59386",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1879210 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1879226 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1879227 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1879229 INFO  (zkCallback-1321-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33d2952a name:ZooKeeperConnection Watcher:127.0.0.1:37503/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1879230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1879230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1879230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1879234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1879234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1879234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1879235 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1879241 INFO  (zkCallback-1322-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4df99c9f name:ZooKeeperConnection Watcher:127.0.0.1:37503/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1879241 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1879242 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1879246 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1879246 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1879259 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:59386/collection1/ and leader is http://127.0.0.1:59386/collection1/
   [junit4]   2> 1879259 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59386
   [junit4]   2> 1879259 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1879259 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1879259 INFO  (coreZkRegister-3779-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:59386_ r:core_node1) [n:127.0.0.1:59386_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1879263 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:59386_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:59386"} current state version: 3
   [junit4]   2> 1879264 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:59386_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:59386"}
   [junit4]   2> 1879265 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1879375 INFO  (zkCallback-1321-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1879377 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1879722 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores/collection1
   [junit4]   2> 1879723 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001
   [junit4]   2> 1879725 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1879735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@51f45894{/,null,AVAILABLE}
   [junit4]   2> 1879736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.ServerConnector Started ServerConnector@e38695{HTTP/1.1}{127.0.0.1:42590}
   [junit4]   2> 1879736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.e.j.s.Server Started @1883082ms
   [junit4]   2> 1879736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:56897/hdfs__localhost_56897__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001_tempDir-002_jetty1, hostPort=42590, hostContext=/, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores}
   [junit4]   2> 1879737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1879737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/'
   [junit4]   2> 1879764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1879764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1879775 INFO  (zkCallback-1323-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56e2c3c3 name:ZooKeeperConnection Watcher:127.0.0.1:37503/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1879775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1879776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1879783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1879783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/solr.xml
   [junit4]   2> 1879801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores
   [junit4]   2> 1879801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoreContainer New CoreContainer 1153561937
   [junit4]   2> 1879801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/]
   [junit4]   2> 1879802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/lib
   [junit4]   2> 1879802 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/lib).
   [junit4]   2> 1879811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1879813 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1879814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1879814 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1879815 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37503/solr
   [junit4]   2> 1879815 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1879815 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1879820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1879822 INFO  (zkCallback-1325-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@515fa292 name:ZooKeeperConnection Watcher:127.0.0.1:37503 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1879825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1879825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1879827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1879829 INFO  (zkCallback-1326-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11774590 name:ZooKeeperConnection Watcher:127.0.0.1:37503/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1879829 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1879837 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1880853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42590_
   [junit4]   2> 1880854 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:42590_
   [junit4]   2> 1880855 INFO  (zkCallback-1321-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1880855 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:59386_) [n:127.0.0.1:59386_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1880856 INFO  (zkCallback-1326-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1880857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1880858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94475563270995977-127.0.0.1:42590_-n_0000000001
   [junit4]   2> 1880860 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94475563270995972-127.0.0.1:59386_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1880905 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1880905 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1880910 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores
   [junit4]   2> 1880912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores/collection1, collection=collection1, configSetProperties=configsetprops.json, shard=, schema=schema.xml, loadOnStartup=true, name=collection1, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores/collection1/, config=solrconfig.xml}
   [junit4]   2> 1880912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores/collection1/
   [junit4]   2> 1880912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1880913 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1880913 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[462C4EE3F1F34FFF]) [n:127.0.0.1:42590_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1880914 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1880914 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1880920 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:42590_",
   [junit4]   2>   "base_url":"http://127.0.0.1:42590"} current state version: 4
   [junit4]   2> 1880923 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:42590_",
   [junit4]   2>   "base_url":"http://127.0.0.1:42590"}
   [junit4]   2> 1880924 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1880924 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1880929 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for external collection collection1
   [junit4]   2> 1880929 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1880930 INFO  (OverseerStateUpdate-94475563270995972-127.0.0.1:59386_-n_0000000000) [n:127.0.0.1:59386_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1880931 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1880931 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1880931 INFO  (zkCallback-1326-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1880933 INFO  (zkCallback-1326-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1881931 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1881932 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1881932 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1881933 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1881933 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1881934 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_462C4EE3F1F34FFF-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1881960 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1881978 INFO  (coreLoadExecutor-3796-thread-1-processing-n:127.0.0.1:42590_) [n:127.0.0.1:42590_ c:collection1   x:collection1] o.a.s.c.SolrConfig c

[...truncated too long message...]

  [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 2275348 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[462C4EE3F1F34FFF]-worker) [n:127.0.0.1:43079_ c:addReplicaColl s:shard1 r:core_node2 x:addReplicaColl_shard1_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 06, 2015 7:44:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene53, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=zh_CN, timezone=CTT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=36655672,total=532152320
   [junit4]   2> NOTE: All tests run in this JVM: [TestRecovery, TestCollationField, TestPHPSerializedResponseWriter, TestSha256AuthenticationProvider, OverriddenZkACLAndCredentialsProvidersTest, BadComponentTest, TriLevelCompositeIdRoutingTest, DistributedIntervalFacetingTest, FieldMutatingUpdateProcessorTest, HdfsLockFactoryTest, TestSchemaNameResource, DistributedQueryComponentOptimizationTest, HdfsSyncSliceTest, RankQueryTest, TestStressReorder, TestFastWriter, SegmentsInfoRequestHandlerTest, TestAuthenticationFramework, TestLRUCache, TestSolrConfigHandlerConcurrent, TestConfigSetImmutable, TermVectorComponentDistributedTest, XmlUpdateRequestHandlerTest, RegexBoostProcessorTest, SystemInfoHandlerTest, TestCloudSchemaless, DistributedQueryComponentCustomSortTest, TestObjectReleaseTracker, TestSearchPerf, DocExpirationUpdateProcessorFactoryTest, TestFuzzyAnalyzedSuggestions, TestScoreJoinQPNoScore, AutoCommitTest, DeleteReplicaTest, TestWriterPerf, TestSweetSpotSimilarityFactory, AssignTest, HighlighterConfigTest, TestPerFieldSimilarity, TestStressRecovery, TestFileDictionaryLookup, StatelessScriptUpdateProcessorFactoryTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestComplexPhraseQParserPlugin, StatsComponentTest, TestRestoreCore, SharedFSAutoReplicaFailoverUtilsTest, TestRandomRequestDistribution, HardAutoCommitTest, TestOverriddenPrefixQueryForCustomFieldType, RAMDirectoryFactoryTest, TestGroupingSearch, TestReload, CoreAdminRequestStatusTest, TestDistribDocBasedVersion, FieldAnalysisRequestHandlerTest, TestInitParams, BasicDistributedZk2Test, NoCacheHeaderTest, TestCloudPivotFacet, UpdateParamsTest, TestDefaultSearchFieldResource, TestRandomMergePolicy, TestSort, TestBlendedInfixSuggestions, SuggesterTSTTest, TestLRUStatsCache, TestHashQParserPlugin, OverseerStatusTest, ShowFileRequestHandlerTest, ZkStateWriterTest, TestZkChroot, RollingRestartTest, CollectionTooManyReplicasTest, BinaryUpdateRequestHandlerTest, TestFastLRUCache, SolrCoreCheckLockOnStartupTest, UUIDFieldTest, TestSolrXml, TestChildDocTransformer, RemoteQueryErrorTest, SharedFSAutoReplicaFailoverTest, TestSolrQueryParserResource, DistributedFacetPivotLargeTest, DirectSolrConnectionTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [275/537] on J2 in 411.66s, 1 test, 1 error <<< FAILURES!

[...truncated 836 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 537 suites, 2124 tests, 1 error, 29 ignored (17 assumptions)

Total time: 191 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1223234328 bytes
Compression is 0.0%
Took 3 min 29 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 949 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/949/

4 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=13268, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13268, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:49704/q_dq/a: collection already exists: awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([E127586EFF534F12]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
252 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=15189, name=RecoveryThread-awholynewstresscollection_collection3_1_shard4_replica1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)    2) Thread[id=14322, name=zkCallback-1188-thread-14, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=13107, name=searcherExecutor-3500-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=14069, name=searcherExecutor-4247-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=14772, name=coreZkRegister-3397-thread-36-processing-r:core_node13 n:127.0.0.1:49704_q_dq%2Fa x:awholynewstresscollection_collection4_1_shard5_replica2 s:shard5 c:awholynewstresscollection_collection4_1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1073)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1043)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:999)         at org.apache.solr.cloud.ZkController.register(ZkController.java:940)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:232)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=13257, name=searcherExecutor-3630-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=12959, name=qtp1109449983-12959-acceptor-0@b0cca0e-ServerConnector@2112016c{HTTP/1.1}{127.0.0.1:33452}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=15363, name=coreZkRegister-3375-thread-36, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=14082, name=searcherExecutor-4265-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=15576, name=OverseerStateUpdate-94469913673531479-127.0.0.1:49704_q_dq%2Fa-n_0000000007, state=TIMED_WAITING, group=Overseer state updater.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:170)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:144)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:193)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=14616, name=zkCallback-1184-thread-10-processing-n:127.0.0.1:58737_q_dq%2Fa-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   12) Thread[id=15558, name=zkCallback-1192-thread-56, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=14060, name=searcherExecutor-4243-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=14020, name=searcherExecutor-4223-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=13860, name=searcherExecutor-4047-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=15033, name=RecoveryThread-collection1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   17) Thread[id=14070, name=searcherExecutor-4277-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=13914, name=searcherExecutor-4131-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=12955, name=Scheduler-434837621, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=15554, name=zkCallback-1188-thread-36, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=12977, name=Thread-7527, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1153)   22) Thread[id=13855, name=searcherExecutor-4025-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=12950, name=searcherExecutor-3403-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=13847, name=searcherExecutor-3998-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=13819, name=searcherExecutor-3976-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=13868, name=searcherExecutor-4051-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=12824, name=Thread-7464, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Thread.join(Thread.java:1281)         at java.lang.Thread.join(Thread.java:1355)         at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)   28) Thread[id=14921, name=qtp1613046501-14921, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=15401, name=zkCallback-1192-thread-51, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=15579, name=RecoveryThread-awholynewcollection_1_shard2_replica1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   31) Thread[id=14938, name=RecoveryThread-awholynewstresscollection_collection1_1_shard2_replica2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   32) Thread[id=14480, name=zkCallback-1197-thread-5-processing-n:127.0.0.1:33452_q_dq%2Fa-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   33) Thread[id=14041, name=searcherExecutor-4226-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=15200, name=RecoveryThread-awholynewstresscollection_collection1_1_shard4_replica1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   35) Thread[id=13034, name=searcherExecutor-3440-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=14476, name=zkCallback-1197-thread-5-processing-n:127.0.0.1:33452_q_dq%2Fa-SendThread(127.0.0.1:58646), state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   37) Thread[id=14081, name=searcherExecutor-4260-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=15172, name=RecoveryThread-awholynewstresscollection_collection4_1_shard2_replica3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   39) Thread[id=15109, name=coreZkRegister-3386-thread-33-processing-r:core_node19 n:127.0.0.1:53380_q_dq%2Fa x:awholynewstresscollection_collection1_1_shard5_replica2 s:shard5 c:awholynewstresscollection_collection1_1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1073)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1043)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:999)         at org.apache.solr.cloud.ZkController.register(ZkController.java:940)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:232)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=15537, name=zkCallback-1188-thread-26, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=15072, name=coreZkRegister-3386-thread-12-processing-r:core_node19 n:127.0.0.1:53380_q_dq%2Fa x:awholynewstresscollection_collection1_1_shard5_replica2 s:shard5 c:awholynewstresscollection_collection1_1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1073)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1043)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:999)         at org.apache.solr.cloud.ZkController.register(ZkController.java:940)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:232)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=14613, name=zkCallback-1192-thread-4-processing-n:127.0.0.1:49704_q_dq%2Fa-SendThread(127.0.0.1:58646), state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   43) Thread[id=15575, name=zkCallback-1192-thread-73, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=13157, name=searcherExecutor-3539-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=15267, name=RecoveryThread-awholynewcollection_4_shard3_replica1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   46) Thread[id=13867, name=searcherExecutor-4046-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=13077, name=searcherExecutor-3478-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=13289, name=qtp210979374-13289, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=12907, name=org.eclipse.jetty.server.session.HashSessionManager@7e31c980Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=14068, name=searcherExecutor-4251-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=13831, name=searcherExecutor-3985-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=13056, name=Scheduler-638982579, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=12958, name=qtp1109449983-12958-selector-ServerConnectorManager@317355e7/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=15547, name=zkCallback-1192-thread-52, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=14061, name=searcherExecutor-4245-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=15552, name=zkCallback-1192-thread-53, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=15532, name=zkCallback-1188-thread-21, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=13040, name=searcherExecutor-3450-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=13892, name=searcherExecutor-4096-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=14072, name=searcherExecutor-4278-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=15531, name=zkCallback-1188-thread-20, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=13844, name=searcherExecutor-3988-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=12894, name=searcherExecutor-3381-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   64) Thread[id=12826, name=SessionTracker, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)   65) Thread[id=14065, name=searcherExecutor-4253-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   66) Thread[id=13813, name=searcherExecutor-3970-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=12918, name=Thread-7500, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1153)   68) Thread[id=15100, name=zkCallback-1192-thread-49, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=14905, name=coreZkRegister-3397-thread-69-processing-r:core_node13 n:127.0.0.1:49704_q_dq%2Fa x:awholynewstresscollection_collection4_1_shard5_replica2 s:shard5 c:awholynewstresscollection_collection4_1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1073)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1043)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:999)         at org.apache.solr.cloud.ZkController.register(ZkController.java:940)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:232)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   70) Thread[id=14071, name=searcherExecutor-4262-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=15546, name=zkCallback-1184-thread-20, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   72) Thread[id=13908, name=searcherExecutor-4115-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   73) Thread[id=13886, name=searcherExecutor-4071-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   74) Thread[id=14615, name=zkCallback-1184-thread-10-processing-n:127.0.0.1:58737_q_dq%2Fa-SendThread(127.0.0.1:58646), state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   75) Thread[id=12926, name=qtp1613046501-12926-selector-ServerConnectorManager@1c98fb4d/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   76) Thread[id=14893, name=coreZkRegister-3397-thread-57-processing-r:core_node13 n:127.0.0.1:49704_q_dq%2Fa x:awholynewstresscollection_collection4_1_shard5_replica2 s:shard5 c:awholynewstresscollection_collection4_1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1073)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1043)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:999)         at org.apache.solr.cloud.ZkController.register(ZkController.java:940)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:232)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   77) Thread[id=14614, name=zkCallback-1192-thread-4-processing-n:127.0.0.1:49704_q_dq%2Fa-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   78) Thread[id=13862, name=searcherExecutor-4041-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   79) Thread[id=13891, name=searcherExecutor-4072-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   80) Thread[id=14058, name=searcherExecutor-4244-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   81) Thread[id=14012, name=searcherExecutor-4225-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   82) Thread[id=12981, name=searcherExecutor-3414-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   83) Thread[id=15555, name=zkCallback-1188-thread-37, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   84) Thread[id=15166, name=RecoveryThread-awholynewstresscollection_collection5_1_shard4_replica4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   85) Thread[id=14015, name=searcherExecutor-4221-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   86) Thread[id=15566, name=zkCallback-1192-thread-64, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   87) Thread[id=13698, name=qtp210979374-13698, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   88) Thread[id=15569, name=zkCallback-1192-thread-67, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   89) Thread[id=12957, name=qtp1109449983-12957-selector-ServerConnectorManager@317355e7/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   90) Thread[id=15541, name=zkCallback-1188-thread-30, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   91) Thread[id=14702, name=coreZkRegister-3397-thread-12-processing-r:core_node13 n:127.0.0.1:49704_q_dq%2Fa x:awholynewstresscollection_collection4_1_shard5_replica2 s:shard5 c:awholynewstresscollection_collection4_1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1073)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1043)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:999)         at org.apache.solr.cloud.ZkController.register(ZkController.java:940)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:232)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   92) Thread[id=15533, name=zkCallback-1188-thread-22, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   93) Thread[id=15534, name=zkCallback-1188-thread-23, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   94) Thread[id=12900, name=qtp458459932-12900-acceptor-0@453f70ff-ServerConnector@aa6af4f{HTTP/1.1}{127.0.0.1:53380}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   95) Thread[id=13256, name=searcherExecutor-3631-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   96) Thread[id=14103, name=qtp1109449983-14103, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   97) Thread[id=13907, name=searcherExecutor-4091-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   98) Thread[id=15565, name=zkCallback-1192-thread-63, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   99) Thread[id=15574, name=zkCallback-1192-thread-72, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   100) Thread[id=15402, name=RecoveryThread-collection1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:511)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)   101) Thread[id=14644, name=qtp458459932-14644, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   102) Thread[id=15540, name=zkCallback-1188-thread-29, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   103) Thread[id=13832, name=searcherExecutor-3987-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   104) Thread[id=14820, name=qtp210979374-14820, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   105) Thread[id=15543, name=zkCallback-1197-thread-14, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   106) Thread[id=15578, name=OverseerHdfsCoreFailoverThread-94469913673531479-127.0.0.1:49704_q_dq%2Fa-n_0000000007, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)         at java.lang.Thread.run(Thread.java:745)   107) Thread[id=13234, name=searcherExecutor-3620-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   108) Thread[id=14377, name=qtp1109449983-14377, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   109) Thread[id=14570, name=qtp1109449983-14570, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   110) Thread[id=15545, name=zkCallback-1184-thread-19, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   111) Thread[id=14042, name=searcherExecutor-4232-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   112) Thread[id=13980, name=searcherExecutor-4199-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   113) Thread[id=15542, name=zkCallback-1188-thread-31, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   114) Thread[id=12965, name=Scheduler-253302253, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   115) Thread[id=15573, name=zkCallback-1192-thread-71, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)       

[...truncated too long message...]

nchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   191) Thread[id=14082, name=searcherExecutor-4265-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   192) Thread[id=15558, name=zkCallback-1192-thread-56, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   193) Thread[id=14060, name=searcherExecutor-4243-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   194) Thread[id=14322, name=zkCallback-1188-thread-14, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   195) Thread[id=13107, name=searcherExecutor-3500-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   196) Thread[id=14069, name=searcherExecutor-4247-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   197) Thread[id=14772, name=coreZkRegister-3397-thread-36, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   198) Thread[id=13257, name=searcherExecutor-3630-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E127586EFF534F12]:0)
   [junit4] Completed [408/537] on J2 in 1562.29s, 1 test, 3 errors <<< FAILURES!

[...truncated 406 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 537 suites, 2124 tests, 2 suite-level errors, 1 error, 1 failure, 442 ignored (17 assumptions)

Total time: 185 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1362627229 bytes
Compression is 0.0%
Took 3 min 44 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 948 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/948/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test

Error Message:
shard1 is not consistent.  Got 1361 from http://127.0.0.1:39719/collection1lastClient and got 706 from http://127.0.0.1:37092/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 1361 from http://127.0.0.1:39719/collection1lastClient and got 706 from http://127.0.0.1:37092/collection1
	at __randomizedtesting.SeedInfo.seed([860E0799691B06A6:E5A3843C7E76B5E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1246)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1225)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:165)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=20796, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20796, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58130/zaml/s: Could not find collection : awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([860E0799691B06A6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10731 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/init-core-data-001
   [junit4]   2> 1626133 INFO  (SUITE-ChaosMonkeySafeLeaderTest-seed#[860E0799691B06A6]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1626133 INFO  (SUITE-ChaosMonkeySafeLeaderTest-seed#[860E0799691B06A6]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1626139 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1626154 INFO  (Thread-51181) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1626154 INFO  (Thread-51181) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1626244 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.ZkTestServer start zk server on port:53322
   [junit4]   2> 1626245 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1626252 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1626260 INFO  (zkCallback-1576-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4cc28344 name:ZooKeeperConnection Watcher:127.0.0.1:53322 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1626260 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1626261 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1626261 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1626274 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1626286 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1626286 INFO  (zkCallback-1577-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33c6417b name:ZooKeeperConnection Watcher:127.0.0.1:53322/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1626287 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1626287 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1626287 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1626290 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1626291 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1626292 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1626294 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1626294 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1626296 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1626297 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1626303 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1626304 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1626305 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1626306 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1626307 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1626307 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1626309 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1626309 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1626310 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1626311 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1626312 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1626312 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1626314 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1626315 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1626316 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1626316 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1626318 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1626318 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1626614 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1
   [junit4]   2> 1626615 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1626617 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29bf615f{/,null,AVAILABLE}
   [junit4]   2> 1626621 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.ServerConnector Started ServerConnector@421fbcb{HTTP/1.1}{127.0.0.1:46689}
   [junit4]   2> 1626621 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.Server Started @1629833ms
   [junit4]   2> 1626621 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/tempDir-001/control/data, hostPort=46689, hostContext=/}
   [junit4]   2> 1626621 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 1626622 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/'
   [junit4]   2> 1626637 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1626637 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1626639 INFO  (zkCallback-1578-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@173bff04 name:ZooKeeperConnection Watcher:127.0.0.1:53322/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1626639 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1626639 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1626640 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1626640 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/solr.xml
   [junit4]   2> 1626667 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores
   [junit4]   2> 1626667 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoreContainer New CoreContainer 1283478100
   [junit4]   2> 1626667 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/]
   [junit4]   2> 1626668 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/lib
   [junit4]   2> 1626668 WARN  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/lib).
   [junit4]   2> 1626676 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1626677 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1626678 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1626678 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1626678 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53322/solr
   [junit4]   2> 1626678 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1626678 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1626679 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1626680 INFO  (zkCallback-1580-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ad888d name:ZooKeeperConnection Watcher:127.0.0.1:53322 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1626681 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1626681 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1626683 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1626684 INFO  (zkCallback-1581-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@64a89232 name:ZooKeeperConnection Watcher:127.0.0.1:53322/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1626684 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1626685 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1626687 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1626689 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1626691 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1626693 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1626695 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1626697 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1626698 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1626699 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1626700 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46689_
   [junit4]   2> 1626700 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46689_
   [junit4]   2> 1626702 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1626703 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1626704 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1626705 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94464686708293636-127.0.0.1:46689_-n_0000000000
   [junit4]   2> 1626705 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46689_
   [junit4]   2> 1626705 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1626707 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer Overseer (id=94464686708293636-127.0.0.1:46689_-n_0000000000) starting
   [junit4]   2> 1626708 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1626713 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1626714 INFO  (OverseerCollectionConfigSetProcessor-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1626714 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1626715 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1626731 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1626732 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1626733 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores
   [junit4]   2> 1626733 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, shard=, collection=control_collection, coreNodeName=, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/, loadOnStartup=true, configSetProperties=configsetprops.json, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1, schema=schema.xml}
   [junit4]   2> 1626734 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/
   [junit4]   2> 1626734 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1626735 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1626735 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1626735 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1626735 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:46689_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1626737 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46689",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:46689_",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1626737 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1626737 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46689",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:46689_",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1626738 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1626738 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1626740 INFO  (zkCallback-1581-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1627737 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1627738 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1627738 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1627738 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1627739 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1627739 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/'
   [junit4]   2> 1627753 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1627759 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1627772 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1627801 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1627802 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1627808 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1628028 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1628030 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1628031 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1628035 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1628053 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1628053 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1628053 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 1628053 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1628054 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2cbdbd31
   [junit4]   2> 1628054 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/data
   [junit4]   2> 1628055 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1628055 WARN  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1628055 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/data/index
   [junit4]   2> 1628055 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@268e2e15
   [junit4]   2> 1628065 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 1628065 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1628073 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1628074 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1628074 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1628074 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1628074 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1628074 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1628075 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1628075 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1628075 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1628093 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/data/
   [junit4]   2> 1628095 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1628096 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1628098 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1628099 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1628103 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/config,/update/json/docs,/admin/plugins,/replication,/update/json,/update/csv,/admin/luke,/admin/mbeans,/admin/threads,/update,/admin/properties,/schema,/admin/file,/admin/segments,/admin/ping,/admin/logging,/admin/system,/get
   [junit4]   2> 1628105 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1628122 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1628122 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1628123 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1628123 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1628124 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@74f12e62
   [junit4]   2> 1628124 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 1628125 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1628125 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@32aae541[collection1] main
   [junit4]   2> 1628125 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1628126 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1628126 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1628126 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1628127 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1628127 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1628127 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1628127 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1628127 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1628128 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1628128 INFO  (searcherExecutor-3040-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@32aae541[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1628129 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1628129 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1628129 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1628129 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511434989398392832
   [junit4]   2> 1628131 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1511434989398392832
   [junit4]   2> 1628131 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1628132 INFO  (coreLoadExecutor-3039-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1628139 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46689 collection:control_collection shard:shard1
   [junit4]   2> 1628140 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1628143 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94464686708293636-core_node1-n_0000000000
   [junit4]   2> 1628145 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1628147 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1628147 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1628147 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46689/collection1/
   [junit4]   2> 1628147 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1628147 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1628150 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46689/collection1/ has no replicas
   [junit4]   2> 1628150 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46689/collection1/ shard1
   [junit4]   2> 1628151 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1628153 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1628155 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46689",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1628237 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1628238 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1628239 INFO  (zkCallback-1583-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b58f060 name:ZooKeeperConnection Watcher:127.0.0.1:53322/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1628239 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1628239 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1628239 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1628242 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1628242 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1628242 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1628250 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1628251 INFO  (zkCallback-1584-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76d1c231 name:ZooKeeperConnection Watcher:127.0.0.1:53322/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1628251 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1628252 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1628257 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1628258 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1628259 INFO  (zkCallback-1581-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1628261 INFO  (zkCallback-1583-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1628306 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46689/collection1/ and leader is http://127.0.0.1:46689/collection1/
   [junit4]   2> 1628306 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46689
   [junit4]   2> 1628306 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1628306 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1628306 INFO  (coreZkRegister-3033-thread-1-processing-s:shard1 c:control_collection r:core_node1 x:collection1 n:127.0.0.1:46689_) [n:127.0.0.1:46689_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1628308 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46689",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:46689_",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 1628309 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46689",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:46689_",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1628309 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1628413 INFO  (zkCallback-1583-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1628413 INFO  (zkCallback-1581-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1628547 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1
   [junit4]   2> 1628548 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001
   [junit4]   2> 1628548 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1628550 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@8c6c867{/,null,AVAILABLE}
   [junit4]   2> 1628550 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.ServerConnector Started ServerConnector@43cb8865{HTTP/1.1}{127.0.0.1:57951}
   [junit4]   2> 1628551 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.e.j.s.Server Started @1631763ms
   [junit4]   2> 1628551 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostPort=57951, hostContext=/}
   [junit4]   2> 1628551 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 1628551 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/'
   [junit4]   2> 1628566 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1628567 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1628568 INFO  (zkCallback-1585-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@357dbd15 name:ZooKeeperConnection Watcher:127.0.0.1:53322/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1628568 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1628569 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1628570 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1628570 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/solr.xml
   [junit4]   2> 1628583 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores
   [junit4]   2> 1628584 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoreContainer New CoreContainer 1459975293
   [junit4]   2> 1628584 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/]
   [junit4]   2> 1628584 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/lib
   [junit4]   2> 1628584 WARN  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/lib).
   [junit4]   2> 1628593 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1628594 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1628595 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1628595 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1628595 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53322/solr
   [junit4]   2> 1628595 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1628595 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1628596 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1628597 INFO  (zkCallback-1587-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b4a4168 name:ZooKeeperConnection Watcher:127.0.0.1:53322 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1628598 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1628598 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1628600 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1628601 INFO  (zkCallback-1588-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5889ab54 name:ZooKeeperConnection Watcher:127.0.0.1:53322/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1628601 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1628608 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1629613 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57951_
   [junit4]   2> 1629613 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57951_
   [junit4]   2> 1629614 INFO  (zkCallback-1583-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1629614 INFO  (zkCallback-1581-thread-1-processing-n:127.0.0.1:46689_) [n:127.0.0.1:46689_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1629614 INFO  (zkCallback-1588-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1629617 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1629617 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94464686708293641-127.0.0.1:57951_-n_0000000001
   [junit4]   2> 1629618 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94464686708293636-127.0.0.1:46689_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1629627 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1629627 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1629628 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores
   [junit4]   2> 1629628 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/, loadOnStartup=true, dataDir=data/, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1, collection=collection1, config=solrconfig.xml, coreNodeName=, name=collection1, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 1629629 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/
   [junit4]   2> 1629629 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1629630 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1629630 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[860E0799691B06A6]) [n:127.0.0.1:57951_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1629630 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1629630 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1629631 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for external collection collection1
   [junit4]   2> 1629632 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1629632 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:57951",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57951_",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1629632 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1629633 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:57951",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57951_",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1629633 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1629633 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1629633 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1629736 INFO  (OverseerStateUpdate-94464686708293636-127.0.0.1:46689_-n_0000000000) [n:127.0.0.1:46689_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1629736 INFO  (zkCallback-1588-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1629737 INFO  (zkCallback-1588-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1630634 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1630634 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1630635 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1630635 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1630635 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1630636 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1630649 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1630655 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1630668 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1630696 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1630697 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1630703 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1630911 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1630912 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1630913 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1630916 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1630933 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1630933 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1630933 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 1630934 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1630934 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2cbdbd31
   [junit4]   2> 1630934 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1630935 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1630935 WARN  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1630935 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1630936 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@25c9006f
   [junit4]   2> 1630947 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest_860E0799691B06A6-001/shard-1-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 1630947 INFO  (coreLoadExecutor-3050-thread-1-processing-n:127.0.0.1:57951_) [n:127.0.0.1:57951_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1630947 INFO  (OldIndexDirectoryCleanupT

[...truncated too long message...]

ce/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150904_210458_857.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150904_210458_857.suites
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 199 minutes 6 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1815009706 bytes
Compression is 0.0%
Took 5 min 33 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 947 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/947/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=4201, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4201, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46343: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([6190DD4EE46CC554]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=40214, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=40214, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:54611, http://127.0.0.1:36876, http://127.0.0.1:43578, http://127.0.0.1:41340, http://127.0.0.1:33973]
	at __randomizedtesting.SeedInfo.seed([6190DD4EE46CC554]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:54611, http://127.0.0.1:36876, http://127.0.0.1:43578, http://127.0.0.1:41340, http://127.0.0.1:33973]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41340
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:570)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 9 more




Build Log:
[...truncated 10713 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/init-core-data-001
   [junit4]   2> 619705 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 619705 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 619753 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 619764 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 619768 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 619787 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_59191_hdfs____.ty0m0h/webapp
   [junit4]   2> 619912 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 620382 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59191
   [junit4]   2> 620451 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 620453 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 620468 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53294_datanode____.ylpt0q/webapp
   [junit4]   2> 620595 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 621023 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53294
   [junit4]   2> 621073 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 621076 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 621092 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42364_datanode____1h2a8q/webapp
   [junit4]   2> 621121 INFO  (IPC Server handler 3 on 34642) [    ] BlockStateChange BLOCK* processReport: from storage DS-31ed619b-178f-480e-9329-e5416a4f8fe5 node DatanodeRegistration(127.0.0.1, datanodeUuid=93d7ca89-3375-4484-80ba-57aa729dbb7a, infoPort=53294, ipcPort=38989, storageInfo=lv=-56;cid=testClusterID;nsid=918709506;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 621123 INFO  (IPC Server handler 3 on 34642) [    ] BlockStateChange BLOCK* processReport: from storage DS-3601162a-d64d-4873-83e0-eafab8373ff4 node DatanodeRegistration(127.0.0.1, datanodeUuid=93d7ca89-3375-4484-80ba-57aa729dbb7a, infoPort=53294, ipcPort=38989, storageInfo=lv=-56;cid=testClusterID;nsid=918709506;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 621221 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 621599 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42364
   [junit4]   2> 621712 INFO  (IPC Server handler 4 on 34642) [    ] BlockStateChange BLOCK* processReport: from storage DS-d653a21d-8c82-4871-affe-a4c4de2383dc node DatanodeRegistration(127.0.0.1, datanodeUuid=8e943091-c894-42fc-ab70-42efe27c0642, infoPort=42364, ipcPort=47014, storageInfo=lv=-56;cid=testClusterID;nsid=918709506;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 621714 INFO  (IPC Server handler 4 on 34642) [    ] BlockStateChange BLOCK* processReport: from storage DS-d6f94011-0712-45aa-8c89-212662106f18 node DatanodeRegistration(127.0.0.1, datanodeUuid=8e943091-c894-42fc-ab70-42efe27c0642, infoPort=42364, ipcPort=47014, storageInfo=lv=-56;cid=testClusterID;nsid=918709506;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 621767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 621768 INFO  (Thread-1471) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 621768 INFO  (Thread-1471) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 621868 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.ZkTestServer start zk server on port:49058
   [junit4]   2> 621868 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 621869 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 621874 INFO  (zkCallback-243-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49e7f976 name:ZooKeeperConnection Watcher:127.0.0.1:49058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 621874 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 621874 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 621874 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 621878 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 621878 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 621880 INFO  (zkCallback-244-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ad936ed name:ZooKeeperConnection Watcher:127.0.0.1:49058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 621880 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 621881 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 621881 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 621883 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 621884 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 621886 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 621888 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 621888 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 621891 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 621891 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 621893 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 621893 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 621895 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 621895 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 621898 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 621898 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 621900 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 621900 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 621902 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 621902 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 621909 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 621910 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 621912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 621912 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 621914 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 621914 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 621916 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 621916 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 621919 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 621920 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 621921 INFO  (zkCallback-245-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8f201c3 name:ZooKeeperConnection Watcher:127.0.0.1:49058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 621921 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 621922 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 621922 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 621924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 621925 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 621926 INFO  (zkCallback-246-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51be973a name:ZooKeeperConnection Watcher:127.0.0.1:49058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 621926 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 621927 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 621927 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 621927 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 621931 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 621931 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 621933 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 621933 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 621936 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 621936 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 621938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 621939 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 621940 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 621941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 621943 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 621943 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 621945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 621945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 621947 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 621947 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 621949 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 621949 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 621952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 621952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 621954 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 621954 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 622271 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores/collection1
   [junit4]   2> 622273 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 622276 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c8ea929{/,null,AVAILABLE}
   [junit4]   2> 622277 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.ServerConnector Started ServerConnector@791347df{HTTP/1.1}{127.0.0.1:60640}
   [junit4]   2> 622277 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.Server Started @625447ms
   [junit4]   2> 622277 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=60640, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores, solr.data.dir=hdfs://localhost:34642/hdfs__localhost_34642__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001_tempDir-002_control_data}
   [junit4]   2> 622278 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 622278 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/'
   [junit4]   2> 622301 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 622302 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 622304 INFO  (zkCallback-247-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2ee01d42 name:ZooKeeperConnection Watcher:127.0.0.1:49058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 622304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 622304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 622306 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 622306 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/solr.xml
   [junit4]   2> 622327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores
   [junit4]   2> 622327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoreContainer New CoreContainer 1603681588
   [junit4]   2> 622327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/]
   [junit4]   2> 622328 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/lib
   [junit4]   2> 622328 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/lib).
   [junit4]   2> 622337 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 622338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 622339 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 622339 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 622339 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49058/solr
   [junit4]   2> 622339 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 622339 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 622340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 622341 INFO  (zkCallback-249-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@436998d8 name:ZooKeeperConnection Watcher:127.0.0.1:49058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 622342 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 622343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 622345 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 622347 INFO  (zkCallback-250-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72cd2624 name:ZooKeeperConnection Watcher:127.0.0.1:49058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 622347 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 622349 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 622351 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 622355 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 622357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 622359 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 622362 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 622364 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 622365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 622367 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 622368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60640_
   [junit4]   2> 622368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60640_
   [junit4]   2> 622370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 622372 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 622373 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 622374 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94459160707203078-127.0.0.1:60640_-n_0000000000
   [junit4]   2> 622374 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60640_
   [junit4]   2> 622375 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 622376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.Overseer Overseer (id=94459160707203078-127.0.0.1:60640_-n_0000000000) starting
   [junit4]   2> 622378 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 622385 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 622386 INFO  (OverseerCollectionConfigSetProcessor-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 622386 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 622387 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 622406 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 622406 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 622407 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores
   [junit4]   2> 622408 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores/collection1, config=solrconfig.xml, schema=schema.xml, shard=, transient=false, collection=control_collection, name=collection1, loadOnStartup=true, coreNodeName=, dataDir=data/, configSetProperties=configsetprops.json}
   [junit4]   2> 622409 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores/collection1/
   [junit4]   2> 622409 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 622410 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 622410 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:60640_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 622411 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 622411 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 622414 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:60640",
   [junit4]   2>   "node_name":"127.0.0.1:60640_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 622415 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 622415 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:60640",
   [junit4]   2>   "node_name":"127.0.0.1:60640_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 622416 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 622417 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 622419 INFO  (zkCallback-250-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 623416 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 623416 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 623417 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 623417 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 623417 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 624216 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores/collection1/'
   [junit4]   2> 624284 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 624299 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 624339 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 624377 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 624379 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 624389 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 624760 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 624771 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 624773 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 624790 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 624799 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 624813 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 624827 WARN  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 624827 WARN  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 624827 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 624828 WARN  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 624828 WARN  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 624831 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 624831 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 624831 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 624831 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:34642/solr_hdfs_home
   [junit4]   2> 624831 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 624832 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 624832 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c780878
   [junit4]   2> 624832 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 624862 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 624862 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 624864 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 624875 WARN  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:34642/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 624875 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 624913 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 624914 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 624914 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.0, noCFSRatio=0.0]
   [junit4]   2> 624981 INFO  (IPC Server handler 3 on 34642) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43161 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-3601162a-d64d-4873-83e0-eafab8373ff4:NORMAL:127.0.0.1:45073|RBW], ReplicaUnderConstruction[[DISK]DS-d6f94011-0712-45aa-8c89-212662106f18:NORMAL:127.0.0.1:43161|RBW]]} size 0
   [junit4]   2> 624985 INFO  (IPC Server handler 2 on 34642) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45073 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-3601162a-d64d-4873-83e0-eafab8373ff4:NORMAL:127.0.0.1:45073|RBW], ReplicaUnderConstruction[[DISK]DS-d6f94011-0712-45aa-8c89-212662106f18:NORMAL:127.0.0.1:43161|RBW]]} size 0
   [junit4]   2> 624990 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b62f8bc0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44522403; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 624990 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 624997 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:34642/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 625000 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 625000 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 625000 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 625001 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 625001 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 625001 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 625001 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 625001 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 625002 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 625005 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 625006 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 625007 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 625009 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 625019 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/update,/config,/update/csv,/admin/mbeans,/admin/file,/admin/properties,/admin/luke,/update/json/docs,/admin/threads,standard,/admin/logging,/admin/plugins,/admin/segments,/admin/ping,/update/json,/get,/admin/system,/replication
   [junit4]   2> 625021 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 625026 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 625026 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 625039 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 625039 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 625042 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=23.13671875, floorSegmentMB=0.2607421875, forceMergeDeletesPctAllowed=20.47632871696354, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6368169294796562
   [junit4]   2> 625069 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b62f8bc0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44522403; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 625070 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 625071 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@157b7159[collection1] main
   [junit4]   2> 625071 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 625073 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 625073 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 625073 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 625074 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 625074 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 625075 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 625075 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 625075 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 625075 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 625076 INFO  (searcherExecutor-994-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@157b7159[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 625082 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 625082 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 625083 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 625083 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511346574789181440
   [junit4]   2> 625084 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1511346574789181440
   [junit4]   2> 625085 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 625086 INFO  (coreLoadExecutor-993-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 625087 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:60640 collection:control_collection shard:shard1
   [junit4]   2> 625088 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 625092 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94459160707203078-core_node1-n_0000000000
   [junit4]   2> 625092 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 625094 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 625094 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 625095 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60640/collection1/
   [junit4]   2> 625095 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 625095 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 625095 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:60640/collection1/ has no replicas
   [junit4]   2> 625095 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60640/collection1/ shard1
   [junit4]   2> 625096 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 625096 INFO  (zkCallback-250-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 625100 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 625103 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:60640",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 625205 INFO  (zkCallback-250-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 625220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 625221 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 625223 INFO  (zkCallback-252-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11d77068 name:ZooKeeperConnection Watcher:127.0.0.1:49058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 625224 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 625224 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 625224 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 625229 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 625253 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:60640/collection1/ and leader is http://127.0.0.1:60640/collection1/
   [junit4]   2> 625253 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60640
   [junit4]   2> 625253 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 625254 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 625254 INFO  (coreZkRegister-987-thread-1-processing-s:shard1 n:127.0.0.1:60640_ c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:60640_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 625258 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60640",
   [junit4]   2>   "node_name":"127.0.0.1:60640_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 3
   [junit4]   2> 625259 INFO  (OverseerStateUpdate-94459160707203078-127.0.0.1:60640_-n_0000000000) [n:127.0.0.1:60640_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60640",
   [junit4]   2>   "node_name":"127.0.0.1:60640_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 625363 INFO  (zkCallback-250-thread-1-processing-n:127.0.0.1:60640_) [n:127.0.0.1:60640_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 625365 INFO  (zkCallback-252-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 625557 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/cores/collection1
   [junit4]   2> 625559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001
   [junit4]   2> 625562 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 625565 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3dd35ef5{/,null,AVAILABLE}
   [junit4]   2> 625566 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.ServerConnector Started ServerConnector@506fcc5e{HTTP/1.1}{127.0.0.1:39153}
   [junit4]   2> 625566 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.e.j.s.Server Started @628736ms
   [junit4]   2> 625566 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/cores, hostContext=/, hostPort=39153, solr.data.dir=hdfs://localhost:34642/hdfs__localhost_34642__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001_tempDir-002_jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 625567 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 625567 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/'
   [junit4]   2> 625583 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 625584 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 625588 INFO  (zkCallback-253-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33caf6d0 name:ZooKeeperConnection Watcher:127.0.0.1:49058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 625588 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 625589 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 625590 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 625590 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/solr.xml
   [junit4]   2> 625607 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/cores
   [junit4]   2> 625607 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoreContainer New CoreContainer 199466844
   [junit4]   2> 625607 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/]
   [junit4]   2> 625607 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/lib
   [junit4]   2> 625608 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_6190DD4EE46CC554-001/shard-1-001/lib).
   [junit4]   2> 625617 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 625618 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 625619 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 625619 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 625619 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49058/solr
   [junit4]   2> 625619 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 625619 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 625620 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 625621 INFO  (zkCallback-255-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62945531 name:ZooKeeperConnection Watcher:127.0.0.1:49058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 625622 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 625622 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 625624 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:39153_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 625626 INFO  (zkCallback-256-thread-1-processing-n:127.0.0.1:39153_) [n:127.0.0.1:39153_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7bb05237 name:ZooKeeperConnection Watcher:127.0.0.1:49058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 625626 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0.0.1:39153_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 625632 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[6190DD4EE46CC554]) [n:127.0

[...truncated too long message...]

bleHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
   [junit4]    > 	... 9 moreThrowable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=40209, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:54611, http://127.0.0.1:36876, http://127.0.0.1:43578, http://127.0.0.1:41340, http://127.0.0.1:33973]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6190DD4EE46CC554]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:54611, http://127.0.0.1:36876, http://127.0.0.1:43578, http://127.0.0.1:41340, http://127.0.0.1:33973]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41340
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:570)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
   [junit4]    > 	... 9 more
   [junit4]   2> 3380535 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6190DD4EE46CC554]-worker) [n:127.0.0.1:36876_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), other_tl1=PostingsFormat(name=Memory doPackFST= false), intDefault=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=LuceneFixedGap), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l_dv=PostingsFormat(name=LuceneFixedGap), text=BlockTreeOrds(blocksize=128), a_i1=PostingsFormat(name=LuceneVarGapDocFreqInterval), _version_=PostingsFormat(name=Memory doPackFST= false), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50), range_facet_l_dv=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=ro, timezone=Pacific/Chatham
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=63061112,total=514850816
   [junit4]   2> NOTE: All tests run in this JVM: [ConcurrentDeleteAndCreateCollectionTest, SolrTestCaseJ4Test, BufferStoreTest, TestCloudInspectUtil, TestDistributedStatsComponentCardinality, ShardSplitTest, BasicFunctionalityTest, TestQuerySenderNoQuery, TestRebalanceLeaders, ShardRoutingTest, ZkCLITest, ResponseHeaderTest, SolrIndexConfigTest, TestStressVersions, BitVectorTest, SecurityConfHandlerTest, TestUpdate, TestCryptoKeys, TestCollapseQParserPlugin, AnalysisErrorHandlingTest, SchemaVersionSpecificBehaviorTest, SaslZkACLProviderTest, StressHdfsTest, CSVRequestHandlerTest, TestDistributedGrouping, TestWordDelimiterFilterFactory, IndexBasedSpellCheckerTest, ChaosMonkeySafeLeaderTest, MultiThreadedOCPTest, CursorMarkTest, TestInitQParser, TestConfigReload, HdfsChaosMonkeySafeLeaderTest, TestMissingGroups, PreAnalyzedUpdateProcessorTest, DistributedFacetPivotLongTailTest, TestClassNameShortening, AddBlockUpdateTest, MinimalSchemaTest, TestLMJelinekMercerSimilarityFactory, TestDocBasedVersionConstraints, FastVectorHighlighterTest, PreAnalyzedFieldTest, TestFreeTextSuggestions, ZkControllerTest, TermVectorComponentTest, TestSuggestSpellingConverter, TestMacros, DistanceUnitsTest, TestCoreContainer, LoggingHandlerTest, TestMiniSolrCloudCluster, TestManagedSchemaFieldResource, TestNonDefinedSimilarityFactory, DistributedTermsComponentTest, DirectSolrSpellCheckerTest, SpellCheckCollatorTest, SolrPluginUtilsTest, TestCodecSupport, TestSolrCoreProperties, DeleteShardTest, OutOfBoxZkACLAndCredentialsProvidersTest, RulesTest, SoftAutoCommitTest, URLClassifyProcessorTest, SuggesterFSTTest, WordBreakSolrSpellCheckerTest, SmileWriterTest, TestExpandComponent, NotRequiredUniqueKeyTest, AtomicUpdatesTest, SimpleMLTQParserTest, CollectionStateFormat2Test, TestReplicationHandlerBackup, TestSolrConfigHandlerCloud, TestConfigOverlay, TestJsonFacets, TestPartialUpdateDeduplication, LeaderFailoverAfterPartitionTest, TestImplicitCoreProperties, CircularListTest, TestFieldTypeResource, TestFastOutputStream, TestLFUCache, AsyncCallRequestStatusResponseTest, BasicDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [266/537] on J1 in 893.44s, 1 test, 1 error <<< FAILURES!

[...truncated 875 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 537 suites, 2124 tests, 2 errors, 29 ignored (17 assumptions)

Total time: 216 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1252819107 bytes
Compression is 0.0%
Took 3 min 37 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 946 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/946/

6 tests failed.
REGRESSION:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#6): CloudJettyRunner [url=http://127.0.0.1:44493/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#6): CloudJettyRunner [url=http://127.0.0.1:44493/collection1]
	at __randomizedtesting.SeedInfo.seed([648DED8BC12B2DE3:ECD9D2516FD7401B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test

Error Message:
expected:<7> but was:<6>

Stack Trace:
java.lang.AssertionError: expected:<7> but was:<6>
	at __randomizedtesting.SeedInfo.seed([648DED8BC12B2DE3:ECD9D2516FD7401B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.BasicDistributedZkTest.testMultipleCollections(BasicDistributedZkTest.java:1026)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:344)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1441264566786,generation=2,filelist=[_v3q.cfe, _v3q.cfs, _v3q.si, _v5x.cfe, _v5x.cfs, _v5x.si, _v5y.cfe, _v5y.cfs, _v5y.si, _v61.fdt, _v61.fdx, _v61.fnm, _v61.nvd, _v61.nvm, _v61.si, _v61_Lucene50_0.doc, _v61_Lucene50_0.tim, _v61_Lucene50_0.tip, _vv6.cfe, _vv6.cfs, _vv6.si, _vvd.cfe, _vvd.cfs, _vvd.si, _vvf.fdt, _vvf.fdx, _vvf.fnm, _vvf.nvd, _vvf.nvm, _vvf.si, _vvf_Lucene50_0.doc, _vvf_Lucene50_0.tim, _vvf_Lucene50_0.tip, _vvh.cfe, _vvh.cfs, _vvh.si, _vvk.cfe, _vvk.cfs, _vvk.si, _vvn.cfe, _vvn.cfs, _vvn.si, _vz3.cfe, _vz3.cfs, _vz3.si, _vz6.cfe, _vz6.cfs, _vz6.si, _vz7.cfe, _vz7.cfs, _vz7.si, _vz8.cfe, _vz8.cfs, _vz8.si, _vza.fdt, _vza.fdx, _vza.fnm, _vza.nvd, _vza.nvm, _vza.si, _vza_Lucene50_0.doc, _vza_Lucene50_0.tim, _vza_Lucene50_0.tip, _vzb.cfe, _vzb.cfs, _vzb.si, _vzc.cfe, _vzc.cfs, _vzc.si, _vzd.cfe, _vzd.cfs, _vzd.si, _vzf.cfe, _vzf.cfs, _vzf.si, _vzg.cfe, _vzg.cfs, _vzg.si, _vzh.cfe, _vzh.cfs, _vzh.si, _vzi.cfe, _vzi.cfs, _vzi.si, _vzj.cfe, _vzj.cfs, _vzj.si, _vzk.cfe, _vzk.cfs, _vzk.si, _vzl.cfe, _vzl.cfs, _vzl.si, _vzm.cfe, _vzm.cfs, _vzm.si, segments_2]}]> but was:<[{indexVersion=1441264566786,generation=3,filelist=[_v63.cfe, _v63.cfs, _v63.si, _vvq.cfe, _vvq.cfs, _vvq.si, _vz3.cfe, _vz3.cfs, _vz3.si, _vz6.cfe, _vz6.cfs, _vz6.si, _vz7.cfe, _vz7.cfs, _vz7.si, _vz8.cfe, _vz8.cfs, _vz8.si, _vza.fdt, _vza.fdx, _vza.fnm, _vza.nvd, _vza.nvm, _vza.si, _vza_Lucene50_0.doc, _vza_Lucene50_0.tim, _vza_Lucene50_0.tip, _vzb.cfe, _vzb.cfs, _vzb.si, _vzc.cfe, _vzc.cfs, _vzc.si, _vzd.cfe, _vzd.cfs, _vzd.si, _vzf.cfe, _vzf.cfs, _vzf.si, _vzg.cfe, _vzg.cfs, _vzg.si, _vzh.cfe, _vzh.cfs, _vzh.si, _vzi.cfe, _vzi.cfs, _vzi.si, _vzj.cfe, _vzj.cfs, _vzj.si, _vzk.cfe, _vzk.cfs, _vzk.si, _vzl.cfe, _vzl.cfs, _vzl.si, _vzm.cfe, _vzm.cfs, _vzm.si, segments_3]}, {indexVersion=1441264566786,generation=2,filelist=[_v3q.cfe, _v3q.cfs, _v3q.si, _v5x.cfe, _v5x.cfs, _v5x.si, _v5y.cfe, _v5y.cfs, _v5y.si, _v61.fdt, _v61.fdx, _v61.fnm, _v61.nvd, _v61.nvm, _v61.si, _v61_Lucene50_0.doc, _v61_Lucene50_0.tim, _v61_Lucene50_0.tip, _vv6.cfe, _vv6.cfs, _vv6.si, _vvd.cfe, _vvd.cfs, _vvd.si, _vvf.fdt, _vvf.fdx, _vvf.fnm, _vvf.nvd, _vvf.nvm, _vvf.si, _vvf_Lucene50_0.doc, _vvf_Lucene50_0.tim, _vvf_Lucene50_0.tip, _vvh.cfe, _vvh.cfs, _vvh.si, _vvk.cfe, _vvk.cfs, _vvk.si, _vvn.cfe, _vvn.cfs, _vvn.si, _vz3.cfe, _vz3.cfs, _vz3.si, _vz6.cfe, _vz6.cfs, _vz6.si, _vz7.cfe, _vz7.cfs, _vz7.si, _vz8.cfe, _vz8.cfs, _vz8.si, _vza.fdt, _vza.fdx, _vza.fnm, _vza.nvd, _vza.nvm, _vza.si, _vza_Lucene50_0.doc, _vza_Lucene50_0.tim, _vza_Lucene50_0.tip, _vzb.cfe, _vzb.cfs, _vzb.si, _vzc.cfe, _vzc.cfs, _vzc.si, _vzd.cfe, _vzd.cfs, _vzd.si, _vzf.cfe, _vzf.cfs, _vzf.si, _vzg.cfe, _vzg.cfs, _vzg.si, _vzh.cfe, _vzh.cfs, _vzh.si, _vzi.cfe, _vzi.cfs, _vzi.si, _vzj.cfe, _vzj.cfs, _vzj.si, _vzk.cfe, _vzk.cfs, _vzk.si, _vzl.cfe, _vzl.cfs, _vzl.si, _vzm.cfe, _vzm.cfs, _vzm.si, segments_2]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1441264566786,generation=2,filelist=[_v3q.cfe, _v3q.cfs, _v3q.si, _v5x.cfe, _v5x.cfs, _v5x.si, _v5y.cfe, _v5y.cfs, _v5y.si, _v61.fdt, _v61.fdx, _v61.fnm, _v61.nvd, _v61.nvm, _v61.si, _v61_Lucene50_0.doc, _v61_Lucene50_0.tim, _v61_Lucene50_0.tip, _vv6.cfe, _vv6.cfs, _vv6.si, _vvd.cfe, _vvd.cfs, _vvd.si, _vvf.fdt, _vvf.fdx, _vvf.fnm, _vvf.nvd, _vvf.nvm, _vvf.si, _vvf_Lucene50_0.doc, _vvf_Lucene50_0.tim, _vvf_Lucene50_0.tip, _vvh.cfe, _vvh.cfs, _vvh.si, _vvk.cfe, _vvk.cfs, _vvk.si, _vvn.cfe, _vvn.cfs, _vvn.si, _vz3.cfe, _vz3.cfs, _vz3.si, _vz6.cfe, _vz6.cfs, _vz6.si, _vz7.cfe, _vz7.cfs, _vz7.si, _vz8.cfe, _vz8.cfs, _vz8.si, _vza.fdt, _vza.fdx, _vza.fnm, _vza.nvd, _vza.nvm, _vza.si, _vza_Lucene50_0.doc, _vza_Lucene50_0.tim, _vza_Lucene50_0.tip, _vzb.cfe, _vzb.cfs, _vzb.si, _vzc.cfe, _vzc.cfs, _vzc.si, _vzd.cfe, _vzd.cfs, _vzd.si, _vzf.cfe, _vzf.cfs, _vzf.si, _vzg.cfe, _vzg.cfs, _vzg.si, _vzh.cfe, _vzh.cfs, _vzh.si, _vzi.cfe, _vzi.cfs, _vzi.si, _vzj.cfe, _vzj.cfs, _vzj.si, _vzk.cfe, _vzk.cfs, _vzk.si, _vzl.cfe, _vzl.cfs, _vzl.si, _vzm.cfe, _vzm.cfs, _vzm.si, segments_2]}]> but was:<[{indexVersion=1441264566786,generation=3,filelist=[_v63.cfe, _v63.cfs, _v63.si, _vvq.cfe, _vvq.cfs, _vvq.si, _vz3.cfe, _vz3.cfs, _vz3.si, _vz6.cfe, _vz6.cfs, _vz6.si, _vz7.cfe, _vz7.cfs, _vz7.si, _vz8.cfe, _vz8.cfs, _vz8.si, _vza.fdt, _vza.fdx, _vza.fnm, _vza.nvd, _vza.nvm, _vza.si, _vza_Lucene50_0.doc, _vza_Lucene50_0.tim, _vza_Lucene50_0.tip, _vzb.cfe, _vzb.cfs, _vzb.si, _vzc.cfe, _vzc.cfs, _vzc.si, _vzd.cfe, _vzd.cfs, _vzd.si, _vzf.cfe, _vzf.cfs, _vzf.si, _vzg.cfe, _vzg.cfs, _vzg.si, _vzh.cfe, _vzh.cfs, _vzh.si, _vzi.cfe, _vzi.cfs, _vzi.si, _vzj.cfe, _vzj.cfs, _vzj.si, _vzk.cfe, _vzk.cfs, _vzk.si, _vzl.cfe, _vzl.cfs, _vzl.si, _vzm.cfe, _vzm.cfs, _vzm.si, segments_3]}, {indexVersion=1441264566786,generation=2,filelist=[_v3q.cfe, _v3q.cfs, _v3q.si, _v5x.cfe, _v5x.cfs, _v5x.si, _v5y.cfe, _v5y.cfs, _v5y.si, _v61.fdt, _v61.fdx, _v61.fnm, _v61.nvd, _v61.nvm, _v61.si, _v61_Lucene50_0.doc, _v61_Lucene50_0.tim, _v61_Lucene50_0.tip, _vv6.cfe, _vv6.cfs, _vv6.si, _vvd.cfe, _vvd.cfs, _vvd.si, _vvf.fdt, _vvf.fdx, _vvf.fnm, _vvf.nvd, _vvf.nvm, _vvf.si, _vvf_Lucene50_0.doc, _vvf_Lucene50_0.tim, _vvf_Lucene50_0.tip, _vvh.cfe, _vvh.cfs, _vvh.si, _vvk.cfe, _vvk.cfs, _vvk.si, _vvn.cfe, _vvn.cfs, _vvn.si, _vz3.cfe, _vz3.cfs, _vz3.si, _vz6.cfe, _vz6.cfs, _vz6.si, _vz7.cfe, _vz7.cfs, _vz7.si, _vz8.cfe, _vz8.cfs, _vz8.si, _vza.fdt, _vza.fdx, _vza.fnm, _vza.nvd, _vza.nvm, _vza.si, _vza_Lucene50_0.doc, _vza_Lucene50_0.tim, _vza_Lucene50_0.tip, _vzb.cfe, _vzb.cfs, _vzb.si, _vzc.cfe, _vzc.cfs, _vzc.si, _vzd.cfe, _vzd.cfs, _vzd.si, _vzf.cfe, _vzf.cfs, _vzf.si, _vzg.cfe, _vzg.cfs, _vzg.si, _vzh.cfe, _vzh.cfs, _vzh.si, _vzi.cfe, _vzi.cfs, _vzi.si, _vzj.cfe, _vzj.cfs, _vzj.si, _vzk.cfe, _vzk.cfs, _vzk.si, _vzl.cfe, _vzl.cfs, _vzl.si, _vzm.cfe, _vzm.cfs, _vzm.si, segments_2]}]>
	at __randomizedtesting.SeedInfo.seed([648DED8BC12B2DE3:415AF6BBB16323E0]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([648DED8BC12B2DE3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.security.BasicAuthIntegrationTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([648DED8BC12B2DE3]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15092, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15092, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:49239/moaso: Could not find collection : awholynewstresscollection_collection5_3
	at __randomizedtesting.SeedInfo.seed([648DED8BC12B2DE3]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10486 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/init-core-data-001
   [junit4]   2> 878638 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 878638 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 878677 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 878685 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 878689 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 878702 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_52114_hdfs____xhwbup/webapp
   [junit4]   2> 878827 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 879332 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52114
   [junit4]   2> 879387 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 879389 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 879402 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_39806_datanode____e2wrun/webapp
   [junit4]   2> 879521 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 879937 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39806
   [junit4]   2> 879983 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 879985 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 879997 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59727_datanode____cc4k1r/webapp
   [junit4]   2> 880051 INFO  (IPC Server handler 3 on 44303) [    ] BlockStateChange BLOCK* processReport: from storage DS-05918c3c-8f67-4d81-98f7-5bf862ab7bd0 node DatanodeRegistration(127.0.0.1, datanodeUuid=63f63b49-407a-437e-88f8-36025577cc77, infoPort=39806, ipcPort=48484, storageInfo=lv=-56;cid=testClusterID;nsid=138847916;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 880052 INFO  (IPC Server handler 3 on 44303) [    ] BlockStateChange BLOCK* processReport: from storage DS-6c862fad-0bfd-4264-a129-d299c6441919 node DatanodeRegistration(127.0.0.1, datanodeUuid=63f63b49-407a-437e-88f8-36025577cc77, infoPort=39806, ipcPort=48484, storageInfo=lv=-56;cid=testClusterID;nsid=138847916;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 880122 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 880694 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[648DED8BC12B2DE3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59727
   [junit4]   2> 880761 INFO  (IPC Server handler 4 on 44303) [    ] BlockStateChange BLOCK* processReport: from storage DS-7becc13d-8424-4c9f-bc98-d9a9341f21cd node DatanodeRegistration(127.0.0.1, datanodeUuid=dc546e23-d174-4d87-80eb-876bc1803d17, infoPort=59727, ipcPort=46662, storageInfo=lv=-56;cid=testClusterID;nsid=138847916;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 880762 INFO  (IPC Server handler 4 on 44303) [    ] BlockStateChange BLOCK* processReport: from storage DS-a6366f1f-acaa-4dd0-aef6-db052d4e1903 node DatanodeRegistration(127.0.0.1, datanodeUuid=dc546e23-d174-4d87-80eb-876bc1803d17, infoPort=59727, ipcPort=46662, storageInfo=lv=-56;cid=testClusterID;nsid=138847916;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 880850 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 880850 INFO  (Thread-6106) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 880850 INFO  (Thread-6106) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 880950 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.ZkTestServer start zk server on port:48853
   [junit4]   2> 880951 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 880951 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 880954 INFO  (zkCallback-944-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3049fc72 name:ZooKeeperConnection Watcher:127.0.0.1:48853 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 880954 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 880954 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 880954 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 880957 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 880961 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 880963 INFO  (zkCallback-945-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e99dfc6 name:ZooKeeperConnection Watcher:127.0.0.1:48853/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 880963 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 880963 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 880963 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 880965 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 880966 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 880968 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 880969 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 880970 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 880972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 880972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 880974 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 880974 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 880976 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 880977 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 880979 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 880979 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 880981 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 880982 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 880983 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 880983 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 880985 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 880985 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 880987 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 880988 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 880989 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 880990 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 880991 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 880992 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 881298 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores/collection1
   [junit4]   2> 881300 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 881302 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@334dbe80{/,null,AVAILABLE}
   [junit4]   2> 881303 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2f18ebea{HTTP/1.1}{127.0.0.1:33264}
   [junit4]   2> 881303 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.Server Started @884339ms
   [junit4]   2> 881303 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=33264, solr.data.dir=hdfs://localhost:44303/hdfs__localhost_44303__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores, hostContext=/}
   [junit4]   2> 881304 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 881304 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/'
   [junit4]   2> 881320 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 881321 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 881323 INFO  (zkCallback-946-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70496179 name:ZooKeeperConnection Watcher:127.0.0.1:48853/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 881324 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 881324 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 881326 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 881326 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/solr.xml
   [junit4]   2> 881366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores
   [junit4]   2> 881366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.CoreContainer New CoreContainer 1899969146
   [junit4]   2> 881366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/]
   [junit4]   2> 881366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/lib
   [junit4]   2> 881366 WARN  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/lib).
   [junit4]   2> 881378 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 881379 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 881380 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 881380 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 881380 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48853/solr
   [junit4]   2> 881380 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 881381 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 881382 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 881385 INFO  (zkCallback-948-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75ca3bff name:ZooKeeperConnection Watcher:127.0.0.1:48853 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 881386 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 881386 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 881388 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 881389 INFO  (zkCallback-949-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d829af8 name:ZooKeeperConnection Watcher:127.0.0.1:48853/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 881390 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 881391 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 881393 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 881397 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 881399 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 881403 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 881405 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 881407 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 881410 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 881411 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 881412 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33264_
   [junit4]   2> 881412 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33264_
   [junit4]   2> 881414 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 881416 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 881417 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 881418 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94454592667058180-127.0.0.1:33264_-n_0000000000
   [junit4]   2> 881418 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33264_
   [junit4]   2> 881418 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 881421 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.Overseer Overseer (id=94454592667058180-127.0.0.1:33264_-n_0000000000) starting
   [junit4]   2> 881423 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 881431 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 881433 INFO  (OverseerCollectionConfigSetProcessor-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 881433 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 881434 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 881451 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 881451 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 881453 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores
   [junit4]   2> 881454 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, coreNodeName=, collection=control_collection, shard=, loadOnStartup=true, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores/collection1, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, name=collection1, dataDir=data/}
   [junit4]   2> 881454 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores/collection1/
   [junit4]   2> 881454 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 881455 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 881456 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [n:127.0.0.1:33264_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 881456 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 881456 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 881462 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 881462 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:33264_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33264",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 881463 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:33264_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33264",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 881464 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 881464 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 881466 INFO  (zkCallback-949-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 882462 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 882463 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 882463 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 882463 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 882464 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 882464 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores/collection1/'
   [junit4]   2> 882482 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 882489 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 882503 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 882532 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 882533 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 882540 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 882782 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 882789 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 882791 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 882799 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 882802 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 882805 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 882806 WARN  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 882806 WARN  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 882806 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 882807 WARN  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 882807 WARN  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 882808 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 882808 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 882808 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 882808 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44303/solr_hdfs_home
   [junit4]   2> 882809 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 882809 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 882809 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1241ba55
   [junit4]   2> 882809 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 882824 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 882824 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 882825 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 882835 WARN  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:44303/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 882835 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 882845 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 882846 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 882849 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 882849 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 882849 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 882850 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 882869 INFO  (IPC Server handler 1 on 44303) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47871 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6c862fad-0bfd-4264-a129-d299c6441919:NORMAL:127.0.0.1:46696|RBW], ReplicaUnderConstruction[[DISK]DS-a6366f1f-acaa-4dd0-aef6-db052d4e1903:NORMAL:127.0.0.1:47871|FINALIZED]]} size 0
   [junit4]   2> 882872 INFO  (IPC Server handler 2 on 44303) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46696 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6c862fad-0bfd-4264-a129-d299c6441919:NORMAL:127.0.0.1:46696|RBW], ReplicaUnderConstruction[[DISK]DS-a6366f1f-acaa-4dd0-aef6-db052d4e1903:NORMAL:127.0.0.1:47871|FINALIZED]]} size 0
   [junit4]   2> 882874 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b74edba9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5dfb72ac); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 882875 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 882876 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:44303/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 882880 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 882880 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 882880 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 882880 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 882881 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 882881 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 882881 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 882881 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 882881 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 882884 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 882885 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 882887 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 882888 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 882891 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/admin/ping,/update/csv,/get,/schema,/config,/replication,/update/json/docs,/admin/plugins,/admin/properties,/admin/luke,/admin/segments,/admin/system,/admin/logging,/admin/threads,/update/json,/admin/mbeans,/admin/file,standard
   [junit4]   2> 882893 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 882893 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 882893 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 882904 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 882904 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 882905 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3314286520778742]
   [junit4]   2> 882911 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b74edba9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5dfb72ac); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 882911 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 882911 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3c1f3bdd[collection1] main
   [junit4]   2> 882911 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 882912 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 882912 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 882913 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 882913 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 882913 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 882913 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 882914 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 882914 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 882914 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 882915 INFO  (searcherExecutor-3062-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3c1f3bdd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 882916 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 882916 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 882916 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 882917 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511273484838240256
   [junit4]   2> 882917 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1511273484838240256
   [junit4]   2> 882917 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 882917 INFO  (coreLoadExecutor-3061-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 882918 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33264 collection:control_collection shard:shard1
   [junit4]   2> 882918 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 882922 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94454592667058180-core_node1-n_0000000000
   [junit4]   2> 882922 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 882924 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 882924 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 882924 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 882924 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33264/collection1/
   [junit4]   2> 882924 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 882925 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33264/collection1/ has no replicas
   [junit4]   2> 882925 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33264/collection1/ shard1
   [junit4]   2> 882925 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 882928 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 882930 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33264",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 882958 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 882958 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 882959 INFO  (zkCallback-951-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73d0d65f name:ZooKeeperConnection Watcher:127.0.0.1:48853/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 882960 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 882960 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 882960 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 882962 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 883033 INFO  (zkCallback-951-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 883033 INFO  (zkCallback-949-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 883081 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33264/collection1/ and leader is http://127.0.0.1:33264/collection1/
   [junit4]   2> 883081 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33264
   [junit4]   2> 883081 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 883081 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 883081 INFO  (coreZkRegister-3055-thread-1-processing-n:127.0.0.1:33264_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:33264_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 883083 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:33264_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"http://127.0.0.1:33264",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 883084 INFO  (OverseerStateUpdate-94454592667058180-127.0.0.1:33264_-n_0000000000) [n:127.0.0.1:33264_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:33264_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"http://127.0.0.1:33264",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 883186 INFO  (zkCallback-949-thread-1-processing-n:127.0.0.1:33264_) [n:127.0.0.1:33264_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 883186 INFO  (zkCallback-951-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 883290 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/shard-1-001/cores/collection1
   [junit4]   2> 883291 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/shard-1-001
   [junit4]   2> 883292 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 883295 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@748ee09f{/,null,AVAILABLE}
   [junit4]   2> 883296 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5da46c4e{HTTP/1.1}{127.0.0.1:50866}
   [junit4]   2> 883296 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.e.j.s.Server Started @886332ms
   [junit4]   2> 883296 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:44303/hdfs__localhost_44303__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=50866, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/shard-1-001/cores}
   [junit4]   2> 883297 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 883297 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_648DED8BC12B2DE3-001/shard-1-001/'
   [junit4]   2> 883314 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[648DED8BC12B2DE3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [jun

[...truncated too long message...]

ead[id=83182, name=searcherExecutor-8120-thread-1, state=WAITING, group=TGRP-BasicAuthIntegrationTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    7) Thread[id=83187, name=searcherExecutor-8124-thread-1, state=WAITING, group=TGRP-BasicAuthIntegrationTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_i_dv=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=Direct), range_facet_l_dv=FST50, intDefault=PostingsFormat(name=LuceneFixedGap), id=FST50, _version_=PostingsFormat(name=LuceneFixedGap), timestamp=PostingsFormat(name=Direct)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=ar_YE, timezone=Etc/GMT-8
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=23,free=316899136,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [IndexSchemaTest, TestLazyCores, TestComponentsName, TestFreeTextSuggestions, TestEmbeddedSolrServerConstructors, DeleteLastCustomShardedReplicaTest, TestExpandComponent, PingRequestHandlerTest, RequiredFieldsTest, TestReplicationHandlerBackup, SimpleCollectionCreateDeleteTest, SuggestComponentTest, FileBasedSpellCheckerTest, TestMiniSolrCloudClusterSSL, BasicDistributedZkTest, TestSolrDynamicMBean, TestDynamicFieldResource, DistributedVersionInfoTest, TestRangeQuery, CoreMergeIndexesAdminHandlerTest, MigrateRouteKeyTest, TestDistributedSearch, HdfsChaosMonkeySafeLeaderTest, TestRandomDVFaceting, TestLuceneMatchVersion, TestPseudoReturnFields, UUIDUpdateProcessorFallbackTest, TestDistribIDF, TestHdfsUpdateLog, OverseerTaskQueueTest, TestJoin, PolyFieldTest, CursorMarkTest, SignatureUpdateProcessorFactoryTest, TestSolrCloudWithKerberosAlt, TestCSVLoader, SecurityConfHandlerTest, TestRealTimeGet, QueryEqualityTest, HdfsNNFailoverTest, ResponseHeaderTest, RequestLoggingTest, TestInitQParser, DisMaxRequestHandlerTest, TestCollationFieldDocValues, TestFieldSortValues, TestMergePolicyConfig, TestFaceting, ResponseLogComponentTest, QueryElevationComponentTest, SynonymTokenizerTest, TestIntervalFaceting, PeerSyncTest, TestOmitPositions, AnalysisAfterCoreReloadTest, TestAnalyzeInfixSuggestions, TestRequestStatusCollectionAPI, SpellingQueryConverterTest, TestMaxScoreQueryParser, StressHdfsTest, TestIBSimilarityFactory, EchoParamsTest, HdfsBasicDistributedZk2Test, DateMathParserTest, JavabinLoaderTest, DistributedSpellCheckComponentTest, ReplicationFactorTest, TestRemoteStreaming, IndexSchemaRuntimeFieldTest, BadCopyFieldTest, ShardSplitTest, TestStandardQParsers, TestRawResponseWriter, ExternalFileFieldSortTest, TestDocumentBuilder, XsltUpdateRequestHandlerTest, HighlighterMaxOffsetTest, ReturnFieldsTest, TestNamedUpdateProcessors, TestSolr4Spatial, TestCustomDocTransformer, TestExactStatsCache, TestSchemaSimilarityResource, CurrencyFieldOpenExchangeTest, TimeZoneUtilsTest, TestRebalanceLeaders, TestRandomFaceting, OverseerTest, LeaderElectionIntegrationTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, TestRecovery, TestHashPartitioner, HardAutoCommitTest, DistributedTermsComponentTest, TestCoreContainer, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, SolrCmdDistributorTest, BadIndexSchemaTest, TestSort, TestFiltering, TestFunctionQuery, TestBadConfig, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, TestExtendedDismaxParser, SuggesterFSTTest, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, TestStressLucene, TestTrie, NoCacheHeaderTest, TestUpdate, TestAtomicUpdateErrorCases, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DebugComponentTest, CacheHeaderTest, TestSurroundQueryParser, LukeRequestHandlerTest, StandardRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, RequestHandlersTest, DocumentBuilderTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TestArbitraryIndexDir, LoggingHandlerTest, RegexBoostProcessorTest, TestJmxIntegration, TestCollationField, UpdateRequestProcessorFactoryTest, JSONWriterTest, QueryParsingTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, SearchHandlerTest, TestBinaryResponseWriter, SOLR749Test, TestQuerySenderListener, AlternateDirectoryTest, CopyFieldTest, SolrIndexConfigTest, BadComponentTest, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, TestConfig, OutputWriterTest, TestPhraseSuggestions, TestCharFilters, TestCodecSupport, TestXIncludeConfig, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestFastWriter, ResourceLoaderTest, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, PreAnalyzedFieldTest, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, DOMUtilTest, TestUtils, SliceStateTest, FileUtilsTest, DistributedIntervalFacetingTest, CollectionStateFormat2Test, ConcurrentDeleteAndCreateCollectionTest, DeleteInactiveReplicaTest, DeleteShardTest, DistributedQueueTest, HttpPartitionTest, LeaderInitiatedRecoveryOnCommitTest, RemoteQueryErrorTest, TestCollectionAPI, TestCryptoKeys, TestRandomRequestDistribution, HdfsWriteToMultipleCollectionsTest, HdfsDirectoryFactoryTest, TestSolrConfigHandler, TestReqParamsAPI, TestSolrConfigHandlerConcurrent, DistributedQueryComponentCustomSortTest, SpatialHeatmapFacetsTest, TestManagedSchema, AnalyticsQueryTest, TestCustomSort, TestHashQParserPlugin, TestSearcherReuse, TestSimpleQParserPlugin, TestSolr4Spatial2, TestStressUserVersions, BJQParserTest, TestJsonRequest, SimpleMLTQParserTest, TestNonDefinedSimilarityFactory, TestExactSharedStatsCache, TestLRUStatsCache, BasicAuthIntegrationTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicAuthIntegrationTest -Dtests.seed=648DED8BC12B2DE3 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_YE -Dtests.timezone=Etc/GMT-8 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | BasicAuthIntegrationTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([648DED8BC12B2DE3]:0)
   [junit4] Completed [536/536] on J0 in 7223.52s, 4 tests, 2 errors <<< FAILURES!

[...truncated 16 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 536 suites, 2126 tests, 1 suite-level error, 2 errors, 3 failures, 29 ignored (17 assumptions)

Total time: 301 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1084245610 bytes
Compression is 0.0%
Took 3 min 14 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 945 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/945/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior

Error Message:
Illegal state, was: down expected:active clusterState:live nodes:[]collections:{c1=DocCollection(c1)={   "shards":{"shard1":{       "parent":null,       "state":"active",       "range":null,       "replicas":{"core_node1":{           "base_url":"http://127.0.0.1/solr",           "node_name":"node1",           "core":"core1",           "roles":"",           "state":"down"}}}},   "router":{"name":"implicit"}}, test=LazyCollectionRef(test)}

Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
  "shards":{"shard1":{
      "parent":null,
      "state":"active",
      "range":null,
      "replicas":{"core_node1":{
          "base_url":"http://127.0.0.1/solr",
          "node_name":"node1",
          "core":"core1",
          "roles":"",
          "state":"down"}}}},
  "router":{"name":"implicit"}}, test=LazyCollectionRef(test)}
	at __randomizedtesting.SeedInfo.seed([11C48C4FF1E78E6F:79DA8FA31377D421]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.OverseerTest.verifyStatus(OverseerTest.java:601)
	at org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1261)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.search.TestSearcherReuse.test

Error Message:
expected same:<Searcher@1935923e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_6(5.4.0):c6)))}> was not:<Searcher@3bea2266[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_6(5.4.0):c6)))}>

Stack Trace:
java.lang.AssertionError: expected same:<Searcher@1935923e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_6(5.4.0):c6)))}> was not:<Searcher@3bea2266[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_6(5.4.0):c6)))}>
	at __randomizedtesting.SeedInfo.seed([11C48C4FF1E78E6F:9990B3955F1BE397]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotSame(Assert.java:641)
	at org.junit.Assert.assertSame(Assert.java:580)
	at org.junit.Assert.assertSame(Assert.java:593)
	at org.apache.solr.search.TestSearcherReuse.assertSearcherHasNotChanged(TestSearcherReuse.java:247)
	at org.apache.solr.search.TestSearcherReuse.test(TestSearcherReuse.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:42276: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42276: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=108548, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=108548, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51931: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([11C48C4FF1E78E6F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10678 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/init-core-data-001
   [junit4]   2> 1394039 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11C48C4FF1E78E6F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1394039 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11C48C4FF1E78E6F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1394044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1394045 INFO  (Thread-37859) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1394045 INFO  (Thread-37859) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1394145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.ZkTestServer start zk server on port:35670
   [junit4]   2> 1394146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1394147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1394150 INFO  (zkCallback-641-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d49b4d1 name:ZooKeeperConnection Watcher:127.0.0.1:35670 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1394150 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1394151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1394151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1394155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1394155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1394157 INFO  (zkCallback-642-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c978b08 name:ZooKeeperConnection Watcher:127.0.0.1:35670/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1394157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1394157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1394158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1394160 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1394163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1394165 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1394167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1394168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1394171 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1394171 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1394173 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1394174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1394176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1394176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1394178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1394179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1394182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1394182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1394185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1394185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1394188 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1394188 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1394190 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1394190 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1394193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1394193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1394195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1394196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1394199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1394200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1394202 INFO  (zkCallback-643-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36cb0d5a name:ZooKeeperConnection Watcher:127.0.0.1:35670 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1394202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1394203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1394203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1394205 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1394206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1394207 INFO  (zkCallback-644-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@9359fbf name:ZooKeeperConnection Watcher:127.0.0.1:35670/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1394207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1394208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1394208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1394208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1394211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1394212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1394214 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1394215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1394217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1394217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1394219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1394219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1394221 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1394221 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1394224 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1394224 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1394227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1394228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1394230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1394231 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1394233 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1394234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1394236 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1394236 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1394238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1394239 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1394583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1
   [junit4]   2> 1394585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1394586 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@52e04893{/,null,AVAILABLE}
   [junit4]   2> 1394593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3b5e5e98{HTTP/1.1}{127.0.0.1:34811}
   [junit4]   2> 1394593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.Server Started @1397520ms
   [junit4]   2> 1394594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores, hostPort=34811, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/tempDir-001/control/data}
   [junit4]   2> 1394594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1394594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/'
   [junit4]   2> 1394614 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1394616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1394617 INFO  (zkCallback-645-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a0c3677 name:ZooKeeperConnection Watcher:127.0.0.1:35670/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1394617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1394617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1394618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1394618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/solr.xml
   [junit4]   2> 1394634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores
   [junit4]   2> 1394635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1502294889
   [junit4]   2> 1394635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/]
   [junit4]   2> 1394635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/lib
   [junit4]   2> 1394635 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/lib).
   [junit4]   2> 1394644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1394646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1394647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1394647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1394647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35670/solr
   [junit4]   2> 1394647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1394647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1394648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1394653 INFO  (zkCallback-647-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1aaff20a name:ZooKeeperConnection Watcher:127.0.0.1:35670 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1394654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1394654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1394656 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1394657 INFO  (zkCallback-648-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58866cb0 name:ZooKeeperConnection Watcher:127.0.0.1:35670/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1394657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1394659 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1394661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1394664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1394666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1394668 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1394670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1394672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1394674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1394675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1394676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34811_
   [junit4]   2> 1394677 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34811_
   [junit4]   2> 1394679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1394680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1394681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1394682 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94447118245494790-127.0.0.1:34811_-n_0000000000
   [junit4]   2> 1394683 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34811_
   [junit4]   2> 1394683 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1394684 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.Overseer Overseer (id=94447118245494790-127.0.0.1:34811_-n_0000000000) starting
   [junit4]   2> 1394686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1394693 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1394693 INFO  (OverseerCollectionConfigSetProcessor-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1394694 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1394694 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1394709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1394709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1394710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores
   [junit4]   2> 1394711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/, dataDir=data/, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1, schema=schema.xml, shard=, loadOnStartup=true, name=collection1, collection=control_collection, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 1394711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/
   [junit4]   2> 1394711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1394713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1394713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [n:127.0.0.1:34811_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1394713 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1394713 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1394715 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34811",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34811_"} current state version: 0
   [junit4]   2> 1394715 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1394715 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34811",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34811_"}
   [junit4]   2> 1394716 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1394716 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1394718 INFO  (zkCallback-648-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1395716 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1395716 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1395717 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1395717 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1395718 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1395718 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/'
   [junit4]   2> 1395737 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1395743 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1395758 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1395806 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1395807 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1395815 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1396082 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1396090 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1396092 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1396101 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1396104 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1396107 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1396107 WARN  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1396107 WARN  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1396108 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1396108 WARN  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1396108 WARN  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1396109 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1396109 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1396110 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1396110 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1396110 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2087955
   [junit4]   2> 1396111 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/data
   [junit4]   2> 1396111 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1396111 WARN  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1396111 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/data/index
   [junit4]   2> 1396111 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=676829948, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1396112 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@6693a5c5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67685d1f; maxCacheMB=0.8447265625 maxMergeSizeMB=0.3681640625)),segFN=segments_1,generation=1}
   [junit4]   2> 1396112 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1396113 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/data/
   [junit4]   2> 1396113 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1396117 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1396118 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1396118 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1396118 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1396118 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1396118 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1396119 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1396119 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1396119 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1396122 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1396123 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1396124 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1396125 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1396129 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/properties,/get,/admin/file,/admin/plugins,/replication,/admin/system,/admin/segments,/update/csv,/admin/logging,/update/json/docs,/update/json,/admin/ping,/update,/schema,standard,/config,/admin/luke,/admin/threads
   [junit4]   2> 1396130 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1396131 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1396131 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1396132 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1396132 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1396132 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@8e2d817
   [junit4]   2> 1396133 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@6693a5c5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67685d1f; maxCacheMB=0.8447265625 maxMergeSizeMB=0.3681640625)),segFN=segments_1,generation=1}
   [junit4]   2> 1396133 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1396133 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@19aee34e[collection1] main
   [junit4]   2> 1396133 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1396134 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1396134 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1396135 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1396135 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1396135 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1396135 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1396135 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1396135 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1396136 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1396136 INFO  (searcherExecutor-2896-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@19aee34e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1396136 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1396136 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1396137 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1396137 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511153894120488960
   [junit4]   2> 1396139 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1511153894120488960
   [junit4]   2> 1396139 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1396139 INFO  (coreLoadExecutor-2895-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1396140 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34811 collection:control_collection shard:shard1
   [junit4]   2> 1396141 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1396144 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94447118245494790-core_node1-n_0000000000
   [junit4]   2> 1396145 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1396147 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1396147 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1396147 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34811/collection1/
   [junit4]   2> 1396147 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1396147 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1396147 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34811/collection1/ has no replicas
   [junit4]   2> 1396147 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34811/collection1/ shard1
   [junit4]   2> 1396147 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1396151 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:34811",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1396215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1396215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1396217 INFO  (zkCallback-650-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d381d4b name:ZooKeeperConnection Watcher:127.0.0.1:35670/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1396217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1396217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1396217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1396219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1396254 INFO  (zkCallback-648-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1396254 INFO  (zkCallback-650-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1396302 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:34811/collection1/ and leader is http://127.0.0.1:34811/collection1/
   [junit4]   2> 1396302 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34811
   [junit4]   2> 1396302 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1396303 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1396303 INFO  (coreZkRegister-2889-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:34811_) [n:127.0.0.1:34811_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1396305 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34811",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34811_"} current state version: 2
   [junit4]   2> 1396305 INFO  (OverseerStateUpdate-94447118245494790-127.0.0.1:34811_-n_0000000000) [n:127.0.0.1:34811_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:34811",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34811_"}
   [junit4]   2> 1396408 INFO  (zkCallback-650-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1396408 INFO  (zkCallback-648-thread-1-processing-n:127.0.0.1:34811_) [n:127.0.0.1:34811_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1396509 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/cores/collection1
   [junit4]   2> 1396510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001
   [junit4]   2> 1396510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1396512 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35aa5097{/,null,AVAILABLE}
   [junit4]   2> 1396512 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@70e12fae{HTTP/1.1}{127.0.0.1:42276}
   [junit4]   2> 1396513 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.e.j.s.Server Started @1399440ms
   [junit4]   2> 1396513 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/cores, hostContext=/, hostPort=42276}
   [junit4]   2> 1396513 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1396513 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/'
   [junit4]   2> 1396528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1396529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1396530 INFO  (zkCallback-651-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@573d1ed4 name:ZooKeeperConnection Watcher:127.0.0.1:35670/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1396531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1396531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1396532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1396532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/solr.xml
   [junit4]   2> 1396545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/cores
   [junit4]   2> 1396546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoreContainer New CoreContainer 243604638
   [junit4]   2> 1396546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/]
   [junit4]   2> 1396546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/lib
   [junit4]   2> 1396546 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11C48C4FF1E78E6F-001/shard-1-001/lib).
   [junit4]   2> 1396554 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1396555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1396555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1396555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1396555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35670/solr
   [junit4]   2> 1396556 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1396556 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11C48C4FF1E78E6F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1396557 INFO  (TEST-Colle

[...truncated too long message...]

lass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4184888 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[11C48C4FF1E78E6F]-worker) [n:127.0.0.1:44508_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 01, 2015 7:50:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=PostingsFormat(name=Memory doPackFST= false), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=Lucene50(blocksize=128), intDefault=PostingsFormat(name=Memory doPackFST= false), a_i1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=BlockTreeOrds(blocksize=128), range_facet_l=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=BlockTreeOrds(blocksize=128), range_facet_i_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), _version_=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=hu, timezone=Europe/Amsterdam
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=182705128,total=534249472
   [junit4]   2> NOTE: All tests run in this JVM: [StressHdfsTest, ResponseLogComponentTest, CoreMergeIndexesAdminHandlerTest, CursorMarkTest, TestCloudManagedSchemaConcurrent, TestAnalyzedSuggestions, TimeZoneUtilsTest, DateFieldTest, TestRemoteStreaming, ResourceLoaderTest, SimpleCollectionCreateDeleteTest, TestMiniSolrCloudClusterSSL, RequiredFieldsTest, TestMaxScoreQueryParser, SpatialHeatmapFacetsTest, QueryEqualityTest, FileUtilsTest, PrimUtilsTest, TestHdfsUpdateLog, OverseerTaskQueueTest, SuggestComponentTest, OpenExchangeRatesOrgProviderTest, TestExpandComponent, TestStressLucene, SynonymTokenizerTest, DistributedDebugComponentTest, TestSchemaSimilarityResource, RequestLoggingTest, TestDistributedSearch, SpatialRPTFieldTypeTest, ReturnFieldsTest, TestReplicationHandlerBackup, TestIntervalFaceting, TestCSVLoader, DocumentAnalysisRequestHandlerTest, SyncSliceTest, TestDynamicFieldResource, TestCollationFieldDocValues, DefaultValueUpdateProcessorTest, TestFieldSortValues, TestSolrCloudWithKerberosAlt, TestSolrQueryParserDefaultOperatorResource, XsltUpdateRequestHandlerTest, TestReplicationHandler, TestSolr4Spatial, TestJettySolrRunner, BJQParserTest, HighlighterMaxOffsetTest, DataDrivenBlockJoinTest, SignatureUpdateProcessorFactoryTest, TestNamedUpdateProcessors, TestMacros, TestDistribIDF, TestConfigReload, IndexSchemaTest, DeleteLastCustomShardedReplicaTest, TestFaceting, TestFieldTypeResource, QueryElevationComponentTest, SOLR749Test, JSONWriterTest, TestRequestStatusCollectionAPI, TestRawResponseWriter, TestCloudInspectUtil, SecurityConfHandlerTest, JavabinLoaderTest, JsonLoaderTest, TestDocBasedVersionConstraints, PolyFieldTest, CollectionsAPIAsyncDistributedZkTest, PathHierarchyTokenizerFactoryTest, TestLazyCores, TestMergePolicyConfig, TestFieldTypeCollectionResource, CurrencyFieldOpenExchangeTest, ReplicationFactorTest, HttpPartitionTest, HdfsChaosMonkeySafeLeaderTest, BasicDistributedZkTest, AnalysisErrorHandlingTest, TestEmbeddedSolrServerConstructors, DistributedFacetPivotLongTailTest, FullHLLTest, IndexSchemaRuntimeFieldTest, SolrCoreTest, TestCollectionAPI, StandardRequestHandlerTest, TestRandomFaceting, ZkSolrClientTest, ShardRoutingCustomTest, TestDistributedGrouping, TestJoin, TestCoreContainer, SolrCmdDistributorTest, BadIndexSchemaTest, TestFunctionQuery, TestBadConfig, DistributedQueryElevationComponentTest, AnalysisAfterCoreReloadTest, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTSTTest, SchemaVersionSpecificBehaviorTest, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, DistanceFunctionTest, CacheHeaderTest, DisMaxRequestHandlerTest, TestQueryTypes, FileBasedSpellCheckerTest, TestValueSourceCache, TestSolrQueryParser, SolrPluginUtilsTest, TestPHPSerializedResponseWriter, TestComponentsName, TestQuerySenderListener, TestQuerySenderNoQuery, SolrIndexConfigTest, TestConfig, TestSolrCoreProperties, TestLuceneMatchVersion, EchoParamsTest, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, DateMathParserTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, TestSolrJ, TestDocumentBuilder, DistributedMLTComponentTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestHighlightDedupGrouping, TestTolerantSearch, AsyncCallRequestStatusResponseTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, DeleteInactiveReplicaTest, DeleteShardTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnCommitTest, MigrateRouteKeyTest, RemoteQueryErrorTest, SolrCloudExampleTest, TestCloudPivotFacet, TestDistribDocBasedVersion, TestShortCircuitedRequests, TriLevelCompositeIdRoutingTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [535/536] on J0 in 483.73s, 1 test, 1 error <<< FAILURES!

[...truncated 17 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 536 suites, 2128 tests, 2 errors, 2 failures, 30 ignored (18 assumptions)

Total time: 179 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1075917479 bytes
Compression is 0.0%
Took 3 min 5 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 944 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/944/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=8449, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8449, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57350: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([E9929A1166DF410]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=19441, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19441, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39331: Could not find collection : awholynewstresscollection_collection1_1
	at __randomizedtesting.SeedInfo.seed([E9929A1166DF410]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10564 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/init-core-data-001
   [junit4]   2> 787225 WARN  (SUITE-CollectionsAPIDistributedZkTest-seed#[E9929A1166DF410]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=20 numCloses=20
   [junit4]   2> 787226 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[E9929A1166DF410]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 787226 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[E9929A1166DF410]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 787231 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 787231 INFO  (Thread-4734) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 787232 INFO  (Thread-4734) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 787331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.ZkTestServer start zk server on port:59269
   [junit4]   2> 787332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 787332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 787335 INFO  (zkCallback-625-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@476092e9 name:ZooKeeperConnection Watcher:127.0.0.1:59269 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 787335 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 787335 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 787335 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 787338 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 787338 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 787340 INFO  (zkCallback-626-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50bcca03 name:ZooKeeperConnection Watcher:127.0.0.1:59269/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 787340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 787340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 787340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 787342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 787344 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 787345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 787347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 787347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 787350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 787351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 787353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 787353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 787355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 787355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 787357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 787357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 787360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 787360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 787362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 787362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 787364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 787364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 787370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 787370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 787372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 787373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 787376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 787376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 787679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1
   [junit4]   2> 787681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 787683 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@660b81b1{/,null,AVAILABLE}
   [junit4]   2> 787685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.ServerConnector Started ServerConnector@33996c11{HTTP/1.1}{127.0.0.1:44576}
   [junit4]   2> 787685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.Server Started @791430ms
   [junit4]   2> 787686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores, hostContext=/, hostPort=44576, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/tempDir-001/control/data}
   [junit4]   2> 787686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 787686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/'
   [junit4]   2> 787703 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 787703 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 787705 INFO  (zkCallback-627-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@284d1057 name:ZooKeeperConnection Watcher:127.0.0.1:59269/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 787705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 787705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 787706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 787706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/solr.xml
   [junit4]   2> 787721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores
   [junit4]   2> 787721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoreContainer New CoreContainer 1585294070
   [junit4]   2> 787721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/]
   [junit4]   2> 787721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/lib
   [junit4]   2> 787721 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/lib).
   [junit4]   2> 787729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 787730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 787731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 787731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 787731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59269/solr
   [junit4]   2> 787731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 787731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 787732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 787733 INFO  (zkCallback-629-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@205ee959 name:ZooKeeperConnection Watcher:127.0.0.1:59269 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 787734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 787734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 787736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 787737 INFO  (zkCallback-630-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1c13a28d name:ZooKeeperConnection Watcher:127.0.0.1:59269/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 787737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 787738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 787740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 787743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 787745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 787747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 787750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 787751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 787753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 787755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 787756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44576_
   [junit4]   2> 787756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44576_
   [junit4]   2> 787758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 787759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 787760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 787761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94441164129697796-127.0.0.1:44576_-n_0000000000
   [junit4]   2> 787762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44576_
   [junit4]   2> 787762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 787764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer Overseer (id=94441164129697796-127.0.0.1:44576_-n_0000000000) starting
   [junit4]   2> 787766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 787774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 787774 INFO  (OverseerCollectionConfigSetProcessor-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 787775 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 787775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 787790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 787791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 787792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores
   [junit4]   2> 787792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, name=collection1, configSetProperties=configsetprops.json, dataDir=data/, loadOnStartup=true, shard=, transient=false, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/, schema=schema.xml}
   [junit4]   2> 787793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/
   [junit4]   2> 787793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 787794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 787794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:44576_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 787794 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 787795 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 787797 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 787797 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 787797 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 787798 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 787798 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 787800 INFO  (zkCallback-630-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 788798 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 788798 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 788799 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 788799 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 788800 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 788800 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/'
   [junit4]   2> 788815 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 788821 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 788851 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 788880 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 788881 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 788888 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 789151 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 789159 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 789161 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 789172 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 789175 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 789178 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 789179 WARN  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 789179 WARN  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 789179 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 789180 WARN  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 789180 WARN  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 789181 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 789182 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 789182 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 789182 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 789182 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7327ae6b
   [junit4]   2> 789183 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/data
   [junit4]   2> 789183 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/data/index/
   [junit4]   2> 789183 WARN  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 789184 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/data/index
   [junit4]   2> 789184 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=21, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.19891877252021894]
   [junit4]   2> 789185 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@57293a0a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@53a1933a),segFN=segments_1,generation=1}
   [junit4]   2> 789185 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 789185 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/data/
   [junit4]   2> 789186 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 789190 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 789190 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 789191 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 789191 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 789191 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 789191 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 789192 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 789192 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 789192 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 789195 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 789196 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 789198 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 789199 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 789203 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,/admin/properties,/schema,/admin/ping,/admin/mbeans,/update/json,/admin/system,/config,/admin/logging,/admin/file,/admin/threads,standard,/replication,/admin/plugins,/update,/admin/luke,/admin/segments,/update/csv,/get
   [junit4]   2> 789204 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 789204 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 789205 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 789205 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 789206 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 789206 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 789206 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@57293a0a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@53a1933a),segFN=segments_1,generation=1}
   [junit4]   2> 789207 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 789207 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7e5a9a8c[collection1] main
   [junit4]   2> 789207 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 789208 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 789208 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 789209 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 789209 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 789209 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 789209 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 789209 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 789210 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 789210 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 789210 INFO  (searcherExecutor-1892-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7e5a9a8c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 789211 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 789211 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 789211 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 789211 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1511058628149248000
   [junit4]   2> 789213 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1511058628149248000
   [junit4]   2> 789213 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 789214 INFO  (coreLoadExecutor-1891-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 789214 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44576 collection:control_collection shard:shard1
   [junit4]   2> 789215 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 789218 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94441164129697796-core_node1-n_0000000000
   [junit4]   2> 789219 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 789221 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 789221 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 789221 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 789221 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44576/collection1/
   [junit4]   2> 789221 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 789221 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44576/collection1/ has no replicas
   [junit4]   2> 789221 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44576/collection1/ shard1
   [junit4]   2> 789221 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 789226 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 789296 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 789297 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789298 INFO  (zkCallback-632-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f0a977c name:ZooKeeperConnection Watcher:127.0.0.1:59269/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 789299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 789302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 789302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 789302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 789302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789303 INFO  (zkCallback-633-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19d7ae53 name:ZooKeeperConnection Watcher:127.0.0.1:59269/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789303 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789304 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 789306 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 789306 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 789307 INFO  (zkCallback-632-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 789307 INFO  (zkCallback-630-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 789326 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44576/collection1/ and leader is http://127.0.0.1:44576/collection1/
   [junit4]   2> 789326 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44576
   [junit4]   2> 789326 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 789326 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 789327 INFO  (coreZkRegister-1885-thread-1-processing-c:control_collection n:127.0.0.1:44576_ s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:44576_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 789328 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 789329 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 789330 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 789433 INFO  (zkCallback-632-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 789433 INFO  (zkCallback-630-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 789589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores/collection1
   [junit4]   2> 789590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001
   [junit4]   2> 789592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 789594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@28c7450d{/,null,AVAILABLE}
   [junit4]   2> 789594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.ServerConnector Started ServerConnector@17781bcd{HTTP/1.1}{127.0.0.1:56183}
   [junit4]   2> 789594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.e.j.s.Server Started @793339ms
   [junit4]   2> 789595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores, hostPort=56183, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/tempDir-001/jetty1, hostContext=/}
   [junit4]   2> 789595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 789595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/'
   [junit4]   2> 789612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 789613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789614 INFO  (zkCallback-634-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27948dc9 name:ZooKeeperConnection Watcher:127.0.0.1:59269/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 789616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 789616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/solr.xml
   [junit4]   2> 789634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores
   [junit4]   2> 789635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoreContainer New CoreContainer 158937962
   [junit4]   2> 789635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/]
   [junit4]   2> 789635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/lib
   [junit4]   2> 789635 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/lib).
   [junit4]   2> 789646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 789648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 789648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 789648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 789648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59269/solr
   [junit4]   2> 789649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 789649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 789650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789651 INFO  (zkCallback-636-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50802073 name:ZooKeeperConnection Watcher:127.0.0.1:59269 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 789654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 789655 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c2b7dee name:ZooKeeperConnection Watcher:127.0.0.1:59269/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 789655 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 789661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 790667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56183_
   [junit4]   2> 790667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56183_
   [junit4]   2> 790669 INFO  (zkCallback-632-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 790669 INFO  (zkCallback-630-thread-1-processing-n:127.0.0.1:44576_) [n:127.0.0.1:44576_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 790669 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 790671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 790672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94441164129697801-127.0.0.1:56183_-n_0000000001
   [junit4]   2> 790673 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94441164129697796-127.0.0.1:44576_-n_0000000000 to know if I could be the leader
   [junit4]   2> 790693 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 790694 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 790695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores
   [junit4]   2> 790696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores/collection1, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores/collection1/, name=collection1, shard=, collection=collection1, schema=schema.xml, loadOnStartup=true, config=solrconfig.xml, configSetProperties=configsetprops.json}
   [junit4]   2> 790696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores/collection1/
   [junit4]   2> 790696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 790698 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 790698 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E9929A1166DF410]) [n:127.0.0.1:56183_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 790698 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 790699 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 790700 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for external collection collection1
   [junit4]   2> 790700 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 790701 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:56183",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:56183_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 790701 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 790701 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:56183",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:56183_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 790701 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 790701 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 790702 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 790804 INFO  (OverseerStateUpdate-94441164129697796-127.0.0.1:44576_-n_0000000000) [n:127.0.0.1:44576_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 790805 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 790806 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 791702 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 791702 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 791703 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 791703 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 791704 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 791704 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E9929A1166DF410-001/shard-1-001/cores/collection1/'
   [junit4]   2> 791735 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 791743 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 791757 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 791786 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 791787 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 791794 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 792074 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 792082 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 792084 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 792108 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 792111 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 792114 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 792115 WARN  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 792115 WARN  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 792115 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 792116 WARN  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 792116 WARN  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 792117 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 792117 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 792118 INFO  (coreLoadExecutor-1902-thread-1-processing-n:127.0.0.1:56183_) [n:127.0.0.1:56183_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apac

[...truncated too long message...]

MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3502775 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[E9929A1166DF410]-worker) [n:127.0.0.1:38393_ c:addReplicaColl s:shard2 r:core_node2 x:addReplicaColl_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 31, 2015 6:35:18 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene53, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=ar_OM, timezone=America/Ojinaga
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=139944368,total=531628032
   [junit4]   2> NOTE: All tests run in this JVM: [TestNonDefinedSimilarityFactory, DocumentBuilderTest, TestCloudManagedSchema, FastVectorHighlighterTest, ClusterStateUpdateTest, DistributedQueryComponentOptimizationTest, TestUpdate, ZkSolrClientTest, SpellPossibilityIteratorTest, DistributedTermsComponentTest, TestFastOutputStream, DistributedFacetPivotSmallTest, TestChildDocTransformer, MBeansHandlerTest, TestFiltering, AliasIntegrationTest, TestConfigSetImmutable, TestConfigOverlay, HLLSerializationTest, DistanceUnitsTest, TestBlobHandler, TestBM25SimilarityFactory, TestQuerySenderNoQuery, TestImplicitCoreProperties, IgnoreCommitOptimizeUpdateProcessorFactoryTest, ShardRoutingTest, ChaosMonkeyNothingIsSafeTest, TestFileDictionaryLookup, TestScoreJoinQPScore, RemoteQueryErrorTest, LoggingHandlerTest, ChaosMonkeySafeLeaderTest, TestScoreJoinQPNoScore, CollectionStateFormat2Test, TestBlendedInfixSuggestions, TestManagedSchemaFieldTypeResource, TestReloadDeadlock, TestDynamicFieldCollectionResource, TestQueryTypes, SolrIndexConfigTest, HdfsSyncSliceTest, SolrCmdDistributorTest, TestTrie, BasicAuthIntegrationTest, CollectionsAPIDistributedZkTest, ZkCLITest, IndexBasedSpellCheckerTest, SuggesterFSTTest, TestSolrDeletionPolicy2, TestRandomMergePolicy, DeleteReplicaTest, TestInitParams, TestDistributedMissingSort, HighlighterTest, SchemaVersionSpecificBehaviorTest, SoftAutoCommitTest, SharedFSAutoReplicaFailoverUtilsTest, AnalyticsQueryTest, DistributedIntervalFacetingTest, TestLMJelinekMercerSimilarityFactory, TestCollapseQParserPlugin, TestDistribDocBasedVersion, NotRequiredUniqueKeyTest, DistributedFacetPivotLargeTest, AtomicUpdatesTest, ZkControllerTest, TestSolrConfigHandlerCloud, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [398/536] on J1 in 1731.90s, 1 test, 1 error <<< FAILURES!

[...truncated 470 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 536 suites, 2128 tests, 2 errors, 29 ignored (17 assumptions)

Total time: 195 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1156931098 bytes
Compression is 0.0%
Took 3 min 13 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 943 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/943/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=66781, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=66781, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34897: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([60F8F76DCB914B33]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11647 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/init-core-data-001
   [junit4]   2> 2945014 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2945014 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2945050 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2945058 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2945061 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2945079 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_52048_hdfs____4xfau9/webapp
   [junit4]   2> 2945194 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2945715 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52048
   [junit4]   2> 2945792 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2945794 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2945807 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54189_datanode____.cujoo6/webapp
   [junit4]   2> 2945918 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2946347 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54189
   [junit4]   2> 2946386 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2946388 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2946406 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41837_datanode____31iq1y/webapp
   [junit4]   2> 2946470 INFO  (IPC Server handler 3 on 47887) [    ] BlockStateChange BLOCK* processReport: from storage DS-f4b6fb3f-0e26-46d6-8209-e3714bf08c36 node DatanodeRegistration(127.0.0.1, datanodeUuid=2aafc85d-93af-46e1-afe7-7760c00b3044, infoPort=54189, ipcPort=43584, storageInfo=lv=-56;cid=testClusterID;nsid=1128999484;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2946471 INFO  (IPC Server handler 3 on 47887) [    ] BlockStateChange BLOCK* processReport: from storage DS-a2bec6d5-b9bf-41ed-ae57-35d9f453a4dd node DatanodeRegistration(127.0.0.1, datanodeUuid=2aafc85d-93af-46e1-afe7-7760c00b3044, infoPort=54189, ipcPort=43584, storageInfo=lv=-56;cid=testClusterID;nsid=1128999484;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2946540 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2947087 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41837
   [junit4]   2> 2947161 INFO  (IPC Server handler 4 on 47887) [    ] BlockStateChange BLOCK* processReport: from storage DS-7a4f477a-960e-431d-ab71-e0f6491976d3 node DatanodeRegistration(127.0.0.1, datanodeUuid=657eaab4-c662-4ead-b90e-0ea18b935761, infoPort=41837, ipcPort=35284, storageInfo=lv=-56;cid=testClusterID;nsid=1128999484;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2947162 INFO  (IPC Server handler 4 on 47887) [    ] BlockStateChange BLOCK* processReport: from storage DS-cf73d05d-0ffe-464a-aa29-98f5da99c81c node DatanodeRegistration(127.0.0.1, datanodeUuid=657eaab4-c662-4ead-b90e-0ea18b935761, infoPort=41837, ipcPort=35284, storageInfo=lv=-56;cid=testClusterID;nsid=1128999484;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2947254 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2947255 INFO  (Thread-57252) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2947255 INFO  (Thread-57252) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2947355 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.ZkTestServer start zk server on port:46050
   [junit4]   2> 2947355 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2947356 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2947358 INFO  (zkCallback-1595-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d702f4d name:ZooKeeperConnection Watcher:127.0.0.1:46050 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2947359 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2947359 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2947359 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2947362 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2947362 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2947363 INFO  (zkCallback-1596-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c7ae251 name:ZooKeeperConnection Watcher:127.0.0.1:46050/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2947363 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2947366 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2947366 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2947368 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2947370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2947371 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2947372 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2947372 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2947374 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2947374 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2947375 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2947376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2947377 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2947377 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2947379 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2947379 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2947381 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2947381 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2947385 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2947385 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2947386 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2947387 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2947388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2947388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2947389 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2947390 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2947391 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2947391 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2947393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2947393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2947395 INFO  (zkCallback-1597-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7817d614 name:ZooKeeperConnection Watcher:127.0.0.1:46050 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2947395 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2947395 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2947396 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2947397 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2947397 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2947398 INFO  (zkCallback-1598-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b258af1 name:ZooKeeperConnection Watcher:127.0.0.1:46050/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2947399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2947399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2947399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 2947399 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 2947401 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 2947401 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 2947403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 2947403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 2947404 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2947405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2947406 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 2947406 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 2947408 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 2947408 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 2947409 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 2947410 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 2947411 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2947411 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2947413 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2947413 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2947415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 2947415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 2947416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 2947416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 2947418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 2947418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 2947705 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores/collection1
   [junit4]   2> 2947707 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2947709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7a34cf40{/,null,AVAILABLE}
   [junit4]   2> 2947709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.ServerConnector Started ServerConnector@523ebdb8{HTTP/1.1}{127.0.0.1:47114}
   [junit4]   2> 2947709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.Server Started @2950802ms
   [junit4]   2> 2947709 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=47114, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores, solr.data.dir=hdfs://localhost:47887/hdfs__localhost_47887__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001_tempDir-002_control_data}
   [junit4]   2> 2947710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2947710 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/'
   [junit4]   2> 2947725 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2947726 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2947727 INFO  (zkCallback-1599-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@646ca58a name:ZooKeeperConnection Watcher:127.0.0.1:46050/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2947727 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2947728 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2947729 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2947729 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/solr.xml
   [junit4]   2> 2947744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores
   [junit4]   2> 2947745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoreContainer New CoreContainer 1172360387
   [junit4]   2> 2947745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/]
   [junit4]   2> 2947745 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/lib
   [junit4]   2> 2947745 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/lib).
   [junit4]   2> 2947772 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2947773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2947774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2947774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2947774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46050/solr
   [junit4]   2> 2947774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2947774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2947775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2947776 INFO  (zkCallback-1601-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10f67cfe name:ZooKeeperConnection Watcher:127.0.0.1:46050 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2947777 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2947777 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2947778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2947779 INFO  (zkCallback-1602-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a45c341 name:ZooKeeperConnection Watcher:127.0.0.1:46050/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2947779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2947780 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2947782 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2947785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2947786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2947788 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2947790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2947792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2947793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2947794 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2947795 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47114_
   [junit4]   2> 2947795 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47114_
   [junit4]   2> 2947797 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2947798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2947800 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2947800 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94436067100131334-127.0.0.1:47114_-n_0000000000
   [junit4]   2> 2947801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47114_
   [junit4]   2> 2947801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2947802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.Overseer Overseer (id=94436067100131334-127.0.0.1:47114_-n_0000000000) starting
   [junit4]   2> 2947804 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2947809 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2947810 INFO  (OverseerCollectionConfigSetProcessor-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2947810 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2947810 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2947825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2947825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2947826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores
   [junit4]   2> 2947827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, name=collection1, shard=, loadOnStartup=true, configSetProperties=configsetprops.json, dataDir=data/, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores/collection1, transient=false, config=solrconfig.xml, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores/collection1/}
   [junit4]   2> 2947827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores/collection1/
   [junit4]   2> 2947827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2947828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2947828 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2947828 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2947828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:47114_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2947830 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:47114_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47114"} current state version: 0
   [junit4]   2> 2947830 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:47114_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47114"}
   [junit4]   2> 2947830 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2947830 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2947831 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2947832 INFO  (zkCallback-1602-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2948831 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2948831 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2948832 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2948832 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2948833 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2948833 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores/collection1/'
   [junit4]   2> 2948851 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2948859 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2948875 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2948908 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2948909 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2948917 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2949250 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2949265 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2949267 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2949279 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2949282 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2949285 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2949286 WARN  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2949286 WARN  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2949286 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2949287 WARN  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2949287 WARN  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2949288 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2949288 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2949289 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 2949289 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:47887/solr_hdfs_home
   [junit4]   2> 2949289 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2949289 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2949289 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@9ae5e59
   [junit4]   2> 2949290 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2949331 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 2949332 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2949333 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 2949346 WARN  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:47887/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2949347 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2949371 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 2949371 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2949372 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@72ed76b
   [junit4]   2> 2949393 INFO  (IPC Server handler 0 on 47887) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54884 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7a4f477a-960e-431d-ab71-e0f6491976d3:NORMAL:127.0.0.1:37185|RBW], ReplicaUnderConstruction[[DISK]DS-a2bec6d5-b9bf-41ed-ae57-35d9f453a4dd:NORMAL:127.0.0.1:54884|FINALIZED]]} size 0
   [junit4]   2> 2949396 INFO  (IPC Server handler 1 on 47887) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:37185 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a2bec6d5-b9bf-41ed-ae57-35d9f453a4dd:NORMAL:127.0.0.1:54884|FINALIZED], ReplicaUnderConstruction[[DISK]DS-cf73d05d-0ffe-464a-aa29-98f5da99c81c:NORMAL:127.0.0.1:37185|FINALIZED]]} size 0
   [junit4]   2> 2949399 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7b971a9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@421e0b30; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2949399 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2949400 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:47887/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 2949404 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2949404 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2949404 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2949404 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2949404 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2949405 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2949405 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2949405 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2949405 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2949408 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2949409 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2949411 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2949412 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2949415 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/admin/logging,/update/json,/update,standard,/update/json/docs,/admin/file,/get,/admin/plugins,/admin/system,/config,/admin/properties,/admin/mbeans,/replication,/admin/segments,/admin/luke,/admin/threads,/admin/ping,/schema
   [junit4]   2> 2949417 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2949418 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2949418 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 2949434 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2949434 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2949436 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=49, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4315325450898574]
   [junit4]   2> 2949442 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7b971a9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@421e0b30; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2949442 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2949442 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@54d3e26b[collection1] main
   [junit4]   2> 2949443 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2949443 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2949443 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2949444 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2949444 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2949444 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2949445 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2949445 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2949445 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2949445 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2949446 INFO  (searcherExecutor-5872-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@54d3e26b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2949448 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2949448 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 2949448 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2949448 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1510977075899531264
   [junit4]   2> 2949448 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1510977075899531264
   [junit4]   2> 2949450 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2949450 INFO  (coreLoadExecutor-5871-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2949451 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:47114 collection:control_collection shard:shard1
   [junit4]   2> 2949452 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2949460 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94436067100131334-core_node1-n_0000000000
   [junit4]   2> 2949461 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2949468 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2949473 INFO  (zkCallback-1602-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2949482 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2949482 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2949483 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47114/collection1/
   [junit4]   2> 2949483 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2949483 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:47114/collection1/ has no replicas
   [junit4]   2> 2949483 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47114/collection1/ shard1
   [junit4]   2> 2949483 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2949487 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47114",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 2949590 INFO  (zkCallback-1602-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2949640 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:47114/collection1/ and leader is http://127.0.0.1:47114/collection1/
   [junit4]   2> 2949640 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47114
   [junit4]   2> 2949641 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2949641 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2949641 INFO  (coreZkRegister-5865-thread-1-processing-n:127.0.0.1:47114_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:47114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2949643 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:47114_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47114",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 3
   [junit4]   2> 2949643 INFO  (OverseerStateUpdate-94436067100131334-127.0.0.1:47114_-n_0000000000) [n:127.0.0.1:47114_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:47114_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47114",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 2949746 INFO  (zkCallback-1602-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2949831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2949831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2949832 INFO  (zkCallback-1604-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@31df8ddf name:ZooKeeperConnection Watcher:127.0.0.1:46050/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2949833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2949833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2949833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2949835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2950112 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/cores/collection1
   [junit4]   2> 2950113 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001
   [junit4]   2> 2950114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2950116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@8572142{/,null,AVAILABLE}
   [junit4]   2> 2950116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5d41ccd{HTTP/1.1}{127.0.0.1:39102}
   [junit4]   2> 2950116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.e.j.s.Server Started @2953209ms
   [junit4]   2> 2950116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/cores, hostPort=39102, hostContext=/, solr.data.dir=hdfs://localhost:47887/hdfs__localhost_47887__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001_tempDir-002_jetty1}
   [junit4]   2> 2950117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2950117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/'
   [junit4]   2> 2950136 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2950137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2950138 INFO  (zkCallback-1605-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8d33157 name:ZooKeeperConnection Watcher:127.0.0.1:46050/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2950138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2950139 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2950140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2950140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/solr.xml
   [junit4]   2> 2950154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/cores
   [junit4]   2> 2950154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoreContainer New CoreContainer 1181301780
   [junit4]   2> 2950154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/]
   [junit4]   2> 2950154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/lib
   [junit4]   2> 2950154 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/lib).
   [junit4]   2> 2950168 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2950169 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2950170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2950170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2950170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46050/solr
   [junit4]   2> 2950170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2950170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2950171 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2950172 INFO  (zkCallback-1607-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77de43b7 name:ZooKeeperConnection Watcher:127.0.0.1:46050 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2950173 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2950173 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2950174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2950175 INFO  (zkCallback-1608-thread-1-processing-n:127.0.0.1:39102_) [n:127.0.0.1:39102_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@677e381e name:ZooKeeperConnection Watcher:127.0.0.1:46050/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2950176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2950180 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2951185 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39102_
   [junit4]   2> 2951186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39102_
   [junit4]   2> 2951187 INFO  (zkCallback-1604-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2951187 INFO  (zkCallback-1602-thread-1-processing-n:127.0.0.1:47114_) [n:127.0.0.1:47114_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2951187 INFO  (zkCallback-1608-thread-1-processing-n:127.0.0.1:39102_) [n:127.0.0.1:39102_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2951189 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2951189 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94436067100131338-127.0.0.1:39102_-n_0000000001
   [junit4]   2> 2951190 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94436067100131334-127.0.0.1:47114_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2951204 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2951204 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2951205 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/cores
   [junit4]   2> 2951206 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, dataDir=data/, collection=collection1, shard=, loadOnStartup=true, schema=schema.xml, config=solrconfig.xml, coreNodeName=, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/cores/collection1, transient=false}
   [junit4]   2> 2951206 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_60F8F76DCB914B33-001/shard-1-001/cores/collection1/
   [junit4]   2> 2951206 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2951208 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2951208 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[60F8F76DCB914B33]) [n:127.0.0.1:39102_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFil

[...truncated too long message...]

Impl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4340208 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[60F8F76DCB914B33]-worker) [n:127.0.0.1:36168_ c:addReplicaColl s:shard1 r:core_node1 x:addReplicaColl_shard1_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 30, 2015 8:37:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {a_i1=FST50, multiDefault=PostingsFormat(name=MockRandom), timestamp=FST50, other_tl1=FST50, id=FSTOrd50, a_t=FST50, range_facet_l_dv=FSTOrd50, _version_=FST50, text=PostingsFormat(name=LuceneFixedGap), intDefault=FST50, range_facet_i_dv=FST50, range_facet_l=FST50}, docValues:{timestamp=DocValuesFormat(name=Lucene50), range_facet_l_dv=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=sk_SK, timezone=America/Indiana/Knox
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=190976072,total=531103744
   [junit4]   2> NOTE: All tests run in this JVM: [SimpleFacetsTest, TestCoreDiscovery, TestStressReorder, StatsComponentTest, BadIndexSchemaTest, UnloadDistributedZkTest, TestExactSharedStatsCache, ClusterStateTest, TestFieldCollectionResource, OverseerRolesTest, TestLRUStatsCache, TestFastWriter, RequestHandlersTest, HdfsWriteToMultipleCollectionsTest, TestSchemaManager, TestRemoveLastDynamicCopyField, SolrCoreCheckLockOnStartupTest, TestCollationField, SSLMigrationTest, QueryResultKeyTest, TestReversedWildcardFilterFactory, TestSolrDeletionPolicy1, HLLUtilTest, HdfsDirectoryTest, DistribJoinFromCollectionTest, SuggesterTest, TestFuzzyAnalyzedSuggestions, RegexBoostProcessorTest, DistributedExpandComponentTest, TestLRUCache, TestCrossCoreJoin, TestManagedSchemaDynamicFieldResource, OverseerTest, TestCharFilters, DistributedQueryComponentCustomSortTest, FieldMutatingUpdateProcessorTest, TestShardHandlerFactory, TestPHPSerializedResponseWriter, DirectUpdateHandlerOptimizeTest, UpdateParamsTest, MergeStrategyTest, TestConfigSetsAPIZkFailure, TestAuthenticationFramework, DebugComponentTest, TestInfoStreamLogging, CollectionReloadTest, TestHighFrequencyDictionaryFactory, DistributedFacetPivotSmallAdvancedTest, ConnectionManagerTest, TestBulkSchemaAPI, TestExceedMaxTermLength, HdfsRecoveryZkTest, TestConfigSetsAPIExclusivity, TestZkChroot, ZkNodePropsTest, TestSort, TestSimpleQParserPlugin, TestSolrQueryParser, SampleTest, TestRandomRequestDistribution, HdfsBasicDistributedZkTest, DocValuesMultiTest, SolrXmlInZkTest, UUIDFieldTest, HdfsLockFactoryTest, TermVectorComponentDistributedTest, TestExtendedDismaxParser, TestCSVResponseWriter, TestManagedSynonymFilterFactory, SearchHandlerTest, TestRestoreCore, TestSweetSpotSimilarityFactory, TestSolrQueryParserResource, NoCacheHeaderTest, TestReplicaProperties, TestRTimerTree, TestAuthorizationFramework, RAMDirectoryFactoryTest, ShowFileRequestHandlerTest, TestDefaultStatsCache, TestDefaultSimilarityFactory, SuggesterTSTTest, HardAutoCommitTest, TestNoOpRegenerator, SolrCloudExampleTest, FieldAnalysisRequestHandlerTest, BasicDistributedZkTest, CollectionsAPIDistributedZkTest, ShardRoutingTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, TestRecovery, TestReload, TestFunctionQuery, BasicFunctionalityTest, DirectUpdateHandlerTest, TestIndexSearcher, SolrIndexSplitterTest, SignatureUpdateProcessorFactoryTest, CoreAdminHandlerTest, DocValuesTest, WordBreakSolrSpellCheckerTest, DefaultValueUpdateProcessorTest, SortByFunctionTest, XsltUpdateRequestHandlerTest, TestQueryUtils, PrimitiveFieldTypeTest, TestOmitPositions, TermsComponentTest, FastVectorHighlighterTest, IndexSchemaTest, JSONWriterTest, UniqFieldsUpdateProcessorFactoryTest, PingRequestHandlerTest, HighlighterConfigTest, TestSolrIndexConfig, BadComponentTest, TestSearchPerf, TestConfig, TestPostingsSolrHighlighter, NotRequiredUniqueKeyTest, TestXIncludeConfig, TestDFRSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest, DeleteLastCustomShardedReplicaTest, DistribDocExpirationUpdateProcessorTest, OverseerCollectionConfigSetProcessorTest, OverseerStatusTest, OverseerTaskQueueTest, RecoveryAfterSoftCommitTest, RemoteQueryErrorTest, ShardSplitTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [536/536] on J2 in 1404.71s, 1 test, 1 error <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 536 suites, 2128 tests, 1 error, 29 ignored (17 assumptions)

Total time: 207 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1241638313 bytes
Compression is 0.0%
Took 3 min 26 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 942 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/942/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
KeeperErrorCode = Session expired for /clusterstate.json

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:476)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:257)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForCollectionToDisappear(AbstractDistribZkTestBase.java:196)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.assertCollectionNotExists(AbstractFullDistribZkTestBase.java:1772)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection(CollectionsAPIDistributedZkTest.java:234)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:171)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=14533, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14533, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:48241/xn_dg/pg: Could not find collection : awholynewstresscollection_collection0_1
	at __randomizedtesting.SeedInfo.seed([1557958C6F5FCB0E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11077 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/init-core-data-001
   [junit4]   2> 2081402 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2081402 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /xn_dg/pg
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2081442 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2081454 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2081458 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2081471 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_46574_hdfs____.kgbmfa/webapp
   [junit4]   2> 2081592 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2082941 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46574
   [junit4]   2> 2083024 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2083026 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2083040 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_50515_datanode____.ei3sl7/webapp
   [junit4]   2> 2083175 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2083663 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50515
   [junit4]   2> 2083747 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2083749 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2083772 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36101_datanode____v7j6za/webapp
   [junit4]   2> 2083826 INFO  (IPC Server handler 3 on 40751) [    ] BlockStateChange BLOCK* processReport: from storage DS-95e3ddbc-a422-4b29-a66c-2af3296a8ca9 node DatanodeRegistration(127.0.0.1, datanodeUuid=2871fe05-24cc-4c61-843e-00c78dd6fcbb, infoPort=50515, ipcPort=55909, storageInfo=lv=-56;cid=testClusterID;nsid=722428458;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2083828 INFO  (IPC Server handler 3 on 40751) [    ] BlockStateChange BLOCK* processReport: from storage DS-a0505997-f74c-49ba-bdb5-146214308f8f node DatanodeRegistration(127.0.0.1, datanodeUuid=2871fe05-24cc-4c61-843e-00c78dd6fcbb, infoPort=50515, ipcPort=55909, storageInfo=lv=-56;cid=testClusterID;nsid=722428458;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2083929 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2084242 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36101
   [junit4]   2> 2084362 INFO  (IPC Server handler 4 on 40751) [    ] BlockStateChange BLOCK* processReport: from storage DS-7a91297d-caf9-4c6e-9847-fc99b5429473 node DatanodeRegistration(127.0.0.1, datanodeUuid=a26cc08f-aff6-472a-89f7-7205d48df5d8, infoPort=36101, ipcPort=39677, storageInfo=lv=-56;cid=testClusterID;nsid=722428458;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2084364 INFO  (IPC Server handler 4 on 40751) [    ] BlockStateChange BLOCK* processReport: from storage DS-13d1c22b-afde-499b-a7b2-b93a0fbedbf3 node DatanodeRegistration(127.0.0.1, datanodeUuid=a26cc08f-aff6-472a-89f7-7205d48df5d8, infoPort=36101, ipcPort=39677, storageInfo=lv=-56;cid=testClusterID;nsid=722428458;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2084432 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2084432 INFO  (Thread-6929) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2084433 INFO  (Thread-6929) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2084532 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.ZkTestServer start zk server on port:37908
   [junit4]   2> 2084533 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2084534 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2084537 INFO  (zkCallback-1407-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1b6f1578 name:ZooKeeperConnection Watcher:127.0.0.1:37908 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2084537 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2084538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2084538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2084543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2084545 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2084545 INFO  (zkCallback-1408-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2929f59c name:ZooKeeperConnection Watcher:127.0.0.1:37908/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2084545 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2084546 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2084546 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2084549 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2084551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2084553 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2084555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2084555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2084559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2084559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2084562 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2084563 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2084565 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2084565 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2084570 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2084570 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2084572 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2084572 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2084575 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2084575 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2084577 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2084577 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2084580 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2084580 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2084583 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2084583 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2084585 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2084585 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2084590 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2084590 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2084592 INFO  (zkCallback-1409-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a17f728 name:ZooKeeperConnection Watcher:127.0.0.1:37908 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2084593 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2084593 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2084593 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2084595 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2084596 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2084597 INFO  (zkCallback-1410-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@662cf756 name:ZooKeeperConnection Watcher:127.0.0.1:37908/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2084598 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2084598 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2084599 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 2084599 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 2084602 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 2084602 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 2084605 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 2084605 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 2084608 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2084608 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2084614 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 2084614 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 2084618 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 2084618 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 2084620 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 2084620 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 2084622 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2084622 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2084624 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2084624 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2084626 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 2084627 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 2084629 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 2084629 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 2084631 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 2084631 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 2084955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores/collection1
   [junit4]   2> 2084957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2084961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@fb7d501{/xn_dg/pg,null,AVAILABLE}
   [junit4]   2> 2084961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6e013cb6{HTTP/1.1}{127.0.0.1:44576}
   [junit4]   2> 2084962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.Server Started @2088221ms
   [junit4]   2> 2084962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores, hostContext=/xn_dg/pg, hostPort=44576, solr.data.dir=hdfs://localhost:40751/hdfs__localhost_40751__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001_tempDir-002_control_data}
   [junit4]   2> 2084962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 2084962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/'
   [junit4]   2> 2084979 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2084980 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2084982 INFO  (zkCallback-1411-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3760a9a7 name:ZooKeeperConnection Watcher:127.0.0.1:37908/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2084983 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2084983 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2084984 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2084984 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/solr.xml
   [junit4]   2> 2084999 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores
   [junit4]   2> 2084999 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1652756395
   [junit4]   2> 2084999 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/]
   [junit4]   2> 2084999 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/lib
   [junit4]   2> 2085000 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/lib).
   [junit4]   2> 2085008 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2085010 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2085010 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2085010 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2085011 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37908/solr
   [junit4]   2> 2085011 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2085011 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2085012 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2085013 INFO  (zkCallback-1413-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@266e1af6 name:ZooKeeperConnection Watcher:127.0.0.1:37908 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2085013 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2085014 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2085017 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2085017 INFO  (zkCallback-1414-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57605eee name:ZooKeeperConnection Watcher:127.0.0.1:37908/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2085017 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2085018 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2085020 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2085023 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2085025 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2085027 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2085029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2085031 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2085032 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2085034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2085035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44576_xn_dg%2Fpg
   [junit4]   2> 2085035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44576_xn_dg%2Fpg
   [junit4]   2> 2085037 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2085038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2085039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2085040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000
   [junit4]   2> 2085041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44576_xn_dg%2Fpg
   [junit4]   2> 2085041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2085042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.Overseer Overseer (id=94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) starting
   [junit4]   2> 2085044 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2085050 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2085051 INFO  (OverseerCollectionConfigSetProcessor-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2085051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2085052 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2085069 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2085069 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2085070 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores
   [junit4]   2> 2085071 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores/collection1/, shard=, coreNodeName=, schema=schema.xml, name=collection1, transient=false, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json}
   [junit4]   2> 2085071 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores/collection1/
   [junit4]   2> 2085071 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2085072 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2085072 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2085077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2085078 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2085078 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_xn_dg%2Fpg",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576/xn_dg/pg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 2085078 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2085079 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_xn_dg%2Fpg",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576/xn_dg/pg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 2085079 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2085080 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2085081 INFO  (zkCallback-1414-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2086079 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2086079 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2086080 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2086080 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2086081 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2086081 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores/collection1/'
   [junit4]   2> 2086104 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2086111 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2086125 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2086156 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2086157 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2086164 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2086446 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2086455 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2086457 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2086468 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2086471 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2086475 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2086475 WARN  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2086476 WARN  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2086476 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2086477 WARN  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2086477 WARN  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2086478 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2086478 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2086478 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 2086479 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:40751/solr_hdfs_home
   [junit4]   2> 2086479 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2086479 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2086479 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@479005d8
   [junit4]   2> 2086480 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2086554 WARN  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 2091560 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 2091560 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2091563 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 2091581 WARN  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:40751/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2091581 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2091599 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 2091599 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2091600 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4760048282954563]
   [junit4]   2> 2091644 INFO  (IPC Server handler 3 on 40751) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45769 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-95e3ddbc-a422-4b29-a66c-2af3296a8ca9:NORMAL:127.0.0.1:44728|RBW], ReplicaUnderConstruction[[DISK]DS-7a91297d-caf9-4c6e-9847-fc99b5429473:NORMAL:127.0.0.1:45769|FINALIZED]]} size 0
   [junit4]   2> 2091654 INFO  (IPC Server handler 4 on 40751) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44728 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-95e3ddbc-a422-4b29-a66c-2af3296a8ca9:NORMAL:127.0.0.1:44728|RBW], ReplicaUnderConstruction[[DISK]DS-7a91297d-caf9-4c6e-9847-fc99b5429473:NORMAL:127.0.0.1:45769|FINALIZED]]} size 0
   [junit4]   2> 2091671 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6e53949 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@779ba3f3; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2091671 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2091674 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:40751/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 2091678 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2091678 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2091679 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2091679 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2091679 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2091679 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2091679 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2091680 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2091680 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2091683 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2091684 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2091686 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2091687 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2091692 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/ping,/admin/system,/config,/admin/segments,/update,/admin/threads,/admin/logging,/admin/properties,/update/json/docs,/admin/file,/schema,/get,/admin/luke,standard,/update/csv,/admin/plugins,/update/json,/admin/mbeans,/replication
   [junit4]   2> 2091694 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2091704 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2091705 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 2091723 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2091723 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2091727 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=20.5380859375, floorSegmentMB=0.5791015625, forceMergeDeletesPctAllowed=0.0032205611775315557, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2091797 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6e53949 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@779ba3f3; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2091798 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2091798 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6905ea1b[collection1] main
   [junit4]   2> 2091798 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2091799 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2091799 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2091800 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2091800 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2091800 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2091801 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2091801 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2091802 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2091802 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2091803 INFO  (searcherExecutor-3210-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6905ea1b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2091806 WARN  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:54227/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:54227 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 2091810 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2091810 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 2091811 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2091811 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1510883443072827392
   [junit4]   2> 2091811 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1510883443072827392
   [junit4]   2> 2091811 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2091812 INFO  (coreLoadExecutor-3209-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2091813 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44576/xn_dg/pg collection:control_collection shard:shard1
   [junit4]   2> 2091814 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2091819 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94430214708854790-core_node1-n_0000000000
   [junit4]   2> 2091820 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2091822 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2091822 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2091824 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2091824 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44576/xn_dg/pg/collection1/
   [junit4]   2> 2091825 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2091825 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44576/xn_dg/pg/collection1/ has no replicas
   [junit4]   2> 2091825 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44576/xn_dg/pg/collection1/ shard1
   [junit4]   2> 2091825 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2091825 INFO  (zkCallback-1414-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2091832 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576/xn_dg/pg",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 2091936 INFO  (zkCallback-1414-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2091983 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44576/xn_dg/pg/collection1/ and leader is http://127.0.0.1:44576/xn_dg/pg/collection1/
   [junit4]   2> 2091983 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44576/xn_dg/pg
   [junit4]   2> 2091984 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2091984 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2091984 INFO  (coreZkRegister-3203-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:44576_xn_dg%2Fpg r:core_node1 c:control_collection) [n:127.0.0.1:44576_xn_dg%2Fpg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2091986 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_xn_dg%2Fpg",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576/xn_dg/pg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 2091987 INFO  (OverseerStateUpdate-94430214708854790-127.0.0.1:44576_xn_dg%2Fpg-n_0000000000) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:44576_xn_dg%2Fpg",
   [junit4]   2>   "base_url":"http://127.0.0.1:44576/xn_dg/pg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 2092082 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2092085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2092086 INFO  (zkCallback-1416-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42ec4ca2 name:ZooKeeperConnection Watcher:127.0.0.1:37908/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2092087 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2092087 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2092087 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2092089 INFO  (zkCallback-1414-thread-1-processing-n:127.0.0.1:44576_xn_dg%2Fpg) [n:127.0.0.1:44576_xn_dg%2Fpg    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2092090 INFO  (zkCallback-1416-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 2092093 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2092388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001/cores/collection1
   [junit4]   2> 2092389 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001
   [junit4]   2> 2092390 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2092392 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e006f40{/xn_dg/pg,null,AVAILABLE}
   [junit4]   2> 2092393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@48f4b854{HTTP/1.1}{127.0.0.1:58976}
   [junit4]   2> 2092394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.e.j.s.Server Started @2095652ms
   [junit4]   2> 2092394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=58976, solr.data.dir=hdfs://localhost:40751/hdfs__localhost_40751__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001_tempDir-002_jetty1, hostContext=/xn_dg/pg, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 2092394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 2092394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001/'
   [junit4]   2> 2092412 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2092413 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2092415 INFO  (zkCallback-1417-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10bd6ec5 name:ZooKeeperConnection Watcher:127.0.0.1:37908/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2092415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2092415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2092417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2092417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001/solr.xml
   [junit4]   2> 2092432 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001/cores
   [junit4]   2> 2092432 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1861869549
   [junit4]   2> 2092432 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001/]
   [junit4]   2> 2092433 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1557958C6F5FCB0E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001/shard-1-001/lib
   [junit4]   2> 209243

[...truncated too long message...]

tured an uncaught exception in thread: Thread[id=38597, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:46618, https://127.0.0.1:33223, https://127.0.0.1:39637, https://127.0.0.1:43788, https://127.0.0.1:50812]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1557958C6F5FCB0E]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:46618, https://127.0.0.1:33223, https://127.0.0.1:39637, https://127.0.0.1:43788, https://127.0.0.1:50812]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:46618: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qn-
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 moreThrowable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=38595, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:46618, https://127.0.0.1:33223, https://127.0.0.1:39637, https://127.0.0.1:43788, https://127.0.0.1:50812]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1557958C6F5FCB0E]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:46618, https://127.0.0.1:33223, https://127.0.0.1:39637, https://127.0.0.1:43788, https://127.0.0.1:50812]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:46618: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qn-
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]   2> 4200748 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[1557958C6F5FCB0E]-worker) [n:127.0.0.1:46618_ c:awholynewstresscollection_collection3_0 s:shard2 r:core_node2 x:awholynewstresscollection_collection3_0_shard2_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_1557958C6F5FCB0E-001
   [junit4]   2> Aug 29, 2015 8:00:43 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_l=FST50, a_t=FST50, range_facet_l_dv=PostingsFormat(name=LuceneFixedGap), intDefault=FST50, other_tl1=FST50, multiDefault=PostingsFormat(name=MockRandom), text=PostingsFormat(name=Asserting), id=PostingsFormat(name=LuceneFixedGap), timestamp=FST50, range_facet_i_dv=FST50, _version_=FST50, a_i1=FST50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=nl_NL, timezone=Pacific/Johnston
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=143910648,total=522190848
   [junit4]   2> NOTE: All tests run in this JVM: [TestBlobHandler, BasicAuthIntegrationTest, ChaosMonkeyNothingIsSafeTest, TestStressVersions, TestSolrConfigHandlerConcurrent, SpellCheckCollatorTest, SpellPossibilityIteratorTest, HdfsThreadLeakTest, SharedFSAutoReplicaFailoverUtilsTest, HLLSerializationTest, BlockDirectoryTest, TestMinMaxOnMultiValuedField, RankQueryTest, SliceStateTest, TestMiniSolrCloudCluster, TestQueryUtils, RollingRestartTest, TestCollapseQParserPlugin, TestBM25SimilarityFactory, ChaosMonkeySafeLeaderTest, TestManagedResourceStorage, AliasIntegrationTest, EnumFieldTest, MinimalSchemaTest, AsyncMigrateRouteKeyTest, TestScoreJoinQPScore, DistanceFunctionTest, HdfsDirectoryFactoryTest, TestDistributedStatsComponentCardinality, DateRangeFieldTest, BitVectorTest, DistributedQueryComponentOptimizationTest, TestJmxMonitoredMap, OutOfBoxZkACLAndCredentialsProvidersTest, DocValuesTest, TestNonDefinedSimilarityFactory, TestCoreContainer, TestObjectReleaseTracker, SolrTestCaseJ4Test, TestUtils, TestDistribDocBasedVersion, IndexBasedSpellCheckerTest, TestLMJelinekMercerSimilarityFactory, SimpleMLTQParserTest, TestRestManager, HdfsCollectionsAPIDistributedZkTest, TestFiltering, ZkCLITest, URLClassifyProcessorTest, SharedFSAutoReplicaFailoverTest, HighlighterTest, TestOverriddenPrefixQueryForCustomFieldType, SegmentsInfoRequestHandlerTest, CollectionTooManyReplicasTest, AtomicUpdatesTest, DistributedTermsComponentTest, TestWordDelimiterFilterFactory, TestOrdValues, MoreLikeThisHandlerTest, TestSchemaNameResource, TestInitParams, SortByFunctionTest, SuggesterFSTTest, ShardRoutingCustomTest, AssignTest, TestSortByMinMaxFunction, TestCloudSchemaless, TestHashPartitioner, OverriddenZkACLAndCredentialsProvidersTest, TestDynamicFieldCollectionResource, LeaderFailoverAfterPartitionTest, DistributedFacetPivotSmallTest, SpellCheckComponentTest, TestCloudPivotFacet, DirectSolrSpellCheckerTest, TestTrackingShardHandlerFactory, TestFastOutputStream, TestJsonFacets, TestCodecSupport, SolrIndexConfigTest, SoftAutoCommitTest, RuleEngineTest, SolrInfoMBeanTest, CoreAdminRequestStatusTest, TestSolrDeletionPolicy2, TestQuerySenderListener, HdfsSyncSliceTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [488/536] on J1 in 774.74s, 1 test, 1 error <<< FAILURES!

[...truncated 159 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 536 suites, 2128 tests, 2 errors, 29 ignored (17 assumptions)

Total time: 183 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1240550167 bytes
Compression is 0.0%
Took 3 min 28 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 941 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/941/

1 tests failed.
REGRESSION:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([E374FCA58FE83D09:59A693DD0CC6D31C]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:765)
	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=id:529&rows=20&version=2.2&start=0&qt=standard
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:758)
	... 40 more




Build Log:
[...truncated 10470 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150828_201611_790.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid19070.hprof ...
   [junit4] Heap dump file created [678895299 bytes in 12.484 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150828_201611_790.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "collection0"
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 284 lines...]
   [junit4] Suite: org.apache.solr.update.HardAutoCommitTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001
   [junit4]   2> 2936086 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2936087 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2936087 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 2936087 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2936087 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2936088 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2936114 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2936132 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2936172 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2936173 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 2936179 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2936503 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2936510 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2936512 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2936516 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2936519 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2936521 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2936521 WARN  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2936522 WARN  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2936522 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2936522 WARN  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2936522 WARN  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2936522 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2936522 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 2936522 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 2936537 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 519397786
   [junit4]   2> 2936537 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 2936537 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
   [junit4]   2> 2936537 WARN  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib).
   [junit4]   2> 2936552 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2936553 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 2936556 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2936557 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2936557 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2936557 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2936560 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, loadOnStartup=true, name=collection1, config=solrconfig.xml, schema=schema.xml, transient=false, dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, instanceDir=collection1, collection=collection1, shard=shard1}
   [junit4]   2> 2936561 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 2936565 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2936565 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2936565 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2936594 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2936629 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2936682 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2936682 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 2936688 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2936966 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2936975 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2936977 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2936981 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2936984 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2936986 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2936986 WARN  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2936986 WARN  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2936987 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2936987 WARN  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2936987 WARN  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2936988 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 2936988 INFO  (coreLoadExecutor-4448-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 2936988 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2936988 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 2936989 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@182c331e
   [junit4]   2> 2936989 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001
   [junit4]   2> 2936989 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/index/
   [junit4]   2> 2936989 WARN  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 2936990 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/index
   [junit4]   2> 2936990 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.710924654929771]
   [junit4]   2> 2936991 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6ed6a2e3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d601dee),segFN=segments_1,generation=1}
   [junit4]   2> 2936991 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2936992 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/
   [junit4]   2> 2936996 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2936996 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2936996 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2936997 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2936997 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2936997 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2936997 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2936997 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2936997 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2936998 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2936998 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2936998 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2937001 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937002 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937003 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937004 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937008 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2937010 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: mltrh,/admin/luke,/admin/properties,spellCheckWithWordbreak,/admin/system,/update/csv,spellCheckCompRH_Direct,lazy,/update/json,spellCheckCompRH1,spellCheckCompRH,spellCheckWithWordbreak_Direct,standard,/update/json/docs,/replication,/admin/file,defaults,dismax,/search-facet-def,/search-facet-invariants,/mlt,/admin/mbeans,/get,/admin/plugins,tvrh,/debug/dump,/terms,/admin/segments,/admin/logging,/config,/update,/admin/threads,mock,/admin/ping,/schema
   [junit4]   2> 2937011 WARN  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2937012 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2937014 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2937014 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2937014 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2937014 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2937015 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=48.4814453125, floorSegmentMB=0.37890625, forceMergeDeletesPctAllowed=6.270205770221818, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21636794348312646
   [junit4]   2> 2937015 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6ed6a2e3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d601dee),segFN=segments_1,generation=1}
   [junit4]   2> 2937015 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2937015 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5b42b4eb[collection1] main
   [junit4]   2> 2937016 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2937016 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2937016 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2937016 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2937022 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2937059 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2937060 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2937060 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2937060 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2937060 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2937060 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2937061 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2937061 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2937061 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2937061 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1510799113922281472
   [junit4]   2> 2937061 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2937061 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2937062 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2937062 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2937063 INFO  (searcherExecutor-4449-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5b42b4eb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2937063 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1510799113922281472
   [junit4]   2> 2937063 INFO  (coreLoadExecutor-4448-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2937065 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2937073 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 2937074 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {deleteByQuery=*:* (-1510799113934864384)} 0 1
   [junit4]   2> 2937077 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 2937077 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2937078 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2937078 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2937103 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2937121 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2937171 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2937171 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 2937178 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2937424 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2937437 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2937439 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2937447 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2937450 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2937452 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2937452 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2937453 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2937453 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2937453 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2937453 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2937454 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 2937454 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 2937455 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/]
   [junit4]   2> 2937455 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@182c331e
   [junit4]   2> 2937455 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/index/
   [junit4]   2> 2937461 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2937461 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2937461 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2937461 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2937462 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2937462 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2937462 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2937462 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2937462 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2937463 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2937463 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2937463 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2937466 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/
   [junit4]   2> 2937467 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937469 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937470 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937471 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2937481 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2937484 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: mltrh,/admin/luke,/admin/properties,spellCheckWithWordbreak,/admin/system,/update/csv,spellCheckCompRH_Direct,lazy,/update/json,spellCheckCompRH1,spellCheckCompRH,spellCheckWithWordbreak_Direct,standard,/update/json/docs,/replication,/admin/file,defaults,dismax,/search-facet-def,/search-facet-invariants,/mlt,/admin/mbeans,/get,/admin/plugins,tvrh,/debug/dump,/terms,/admin/segments,/admin/logging,/config,/update,/admin/threads,mock,/admin/ping,/schema
   [junit4]   2> 2937485 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2937486 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2937486 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2937486 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2937487 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4a183ed[collection1] main
   [junit4]   2> 2937487 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2937487 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2937487 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2937487 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2937488 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2937491 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2937492 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2937493 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2937493 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2937495 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1510799113934864384
   [junit4]   2> 2937495 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Creating new IndexWriter...
   [junit4]   2> 2937495 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Waiting until IndexWriter is unused... core=collection1
   [junit4]   2> 2937495 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Closing old IndexWriter... core=collection1
   [junit4]   2> 2937496 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=4, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=64.92578125, floorSegmentMB=1.296875, forceMergeDeletesPctAllowed=1.380333649320633, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8640820706545277
   [junit4]   2> 2937497 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2937498 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2937498 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2937498 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2937498 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2937499 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2937499 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2937499 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2937500 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2937501 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4a183ed[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2937501 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6ed6a2e3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d601dee),segFN=segments_1,generation=1}
   [junit4]   2> 2937505 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2937505 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 2937505 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@29631cd8[collection1] main
   [junit4]   2> 2937505 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.CoreContainer replacing core: collection1
   [junit4]   2> 2937505 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@28b4b6a1
   [junit4]   2> 2937506 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 2937506 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2937525 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] {add=[529 (1510799114399383552)]} 0 9
   [junit4]   2> 2937525 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&rows=20&version=2.2&start=0&qt=standard} hits=0 status=0 QTime=0 
   [junit4]   2> 2937535 INFO  (searcherExecutor-4454-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@29631cd8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2937789 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] {add=[530 (1510799114685644800)]} 0 273
   [junit4]   2> 2937790 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:530&rows=20&version=2.2&start=0&qt=standard} hits=0 status=0 QTime=0 
   [junit4]   2> 2937791 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] {delete=[529 (-1510799114687741952)]} 0 275
   [junit4]   2> 2937792 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&rows=20&version=2.2&start=0&qt=standard} hits=0 status=0 QTime=0 
   [junit4]   2> 2937792 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&rows=20&version=2.2&start=0&qt=standard
   [junit4]   2> 2937793 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&rows=20&version=2.2&start=0&qt=standard:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&rows=20&version=2.2&start=0&qt=standard
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:758)
   [junit4]   2> 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2937794 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[E374FCA58FE83D09]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HardAutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=E374FCA58FE83D09 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_BO -Dtests.timezone=America/Rosario -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.74s J2 | HardAutoCommitTest.testCommitWithin <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E374FCA58FE83D09:59A693DD0CC6D31C]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:765)
   [junit4]    > 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > 	request was:q=id:529&rows=20&version=2.2&start=0&qt=standard
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:758)
   [junit4]    > 	... 40 more
   [junit4]   2> 2937800 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2937800 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=519397786
   [junit4]   2> 2937801 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6d4776fc
   [junit4]   2> 2937801 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=2,adds=2,deletesById=1,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 2937802 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2937802 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2937802 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2937802 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.u.UpdateHandler Committing on IndexWriter close.
   [junit4]   2> 2937804 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6ed6a2e3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d601dee),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6ed6a2e3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d601dee),segFN=segments_2,generation=2}
   [junit4]   2> 2937804 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 2937805 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2937842 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2937843 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001;done=false>>]
   [junit4]   2> 2937843 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001
   [junit4]   2> 2937843 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 2937843 INFO  (SUITE-HardAutoCommitTest-seed#[E374FCA58FE83D09]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001/init-core-data-001/index
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_E374FCA58FE83D09-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_BO, timezone=America/Rosario
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=308939376,total=470286336
   [junit4]   2> NOTE: All tests run in this JVM: [AutoCommitTest, TestCollationField, TestCSVResponseWriter, SampleTest, SystemInfoHandlerTest, DocExpirationUpdateProcessorFactoryTest, FacetPivotSmallTest, FieldAnalysisRequestHandlerTest, DistributedSuggestComponentTest, StatsComponentTest, TestNRTOpen, TestDefaultStatsCache, TestAddFieldRealTimeGet, TestSearcherReuse, LukeRequestHandlerTest, FieldMutatingUpdateProcessorTest, ExitableDirectoryReaderTest, TestSolrQueryParser, TestCoreDiscovery, TestJmxIntegration, TestSolrDeletionPolicy1, ChangedSchemaMergeTest, HdfsRecoveryZkTest, BigEndianAscendingWordDeserializerTest, TestDynamicLoading, TestStressRecovery, DistribJoinFromCollectionTest, CurrencyFieldXmlFileTest, MergeStrategyTest, TestCharFilters, ShowFileRequestHandlerTest, TestWriterPerf, ZkNodePropsTest, TestFuzzyAnalyzedSuggestions, BasicDistributedZk2Test, TestFieldCollectionResource, TestAuthorizationFramework, SimpleFacetsTest, TestClusterStateMutator, DistributedFacetPivotWhiteBoxTest, OverseerStatusTest, TestAuthenticationFramework, TestPivotHelperCode, SuggesterWFSTTest, TestBulkSchemaConcurrent, SuggesterTest, HighlighterConfigTest, TestManagedSchemaDynamicFieldResource, TestConfigSetsAPIExclusivity, TestManagedSynonymFilterFactory, DirectUpdateHandlerOptimizeTest, SuggesterTSTTest, FullSolrCloudDistribCmdsTest, HdfsDirectoryTest, RegexBoostProcessorTest, DistributedQueryComponentCustomSortTest, DistributedExpandComponentTest, TestDefaultSimilarityFactory, TestShardHandlerFactory, UnloadDistributedZkTest, TestShortCircuitedRequests, TestReload, TestSweetSpotSimilarityFactory, TestTolerantSearch, TestRemoveLastDynamicCopyField, TestCursorMarkWithoutUniqueKey, SolrCoreCheckLockOnStartupTest, TestPHPSerializedResponseWriter, HdfsBasicDistributedZkTest, DocValuesMultiTest, ZkStateReaderTest, CleanupOldIndexTest, TestSortingResponseWriter, TestJsonRequest, QueryResultKeyTest, TestPostingsSolrHighlighter, HdfsUnloadDistributedZkTest, DirectSolrConnectionTest, TestLRUCache, TestSearchPerf, TestCrossCoreJoin, TestRTimerTree, TestSystemIdResolver, TestFastLRUCache, QueryParsingTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, OpenCloseCoreStressTest, OverseerTest, LeaderElectionIntegrationTest, RecoveryZkTest, LeaderElectionTest, ZkCLITest, ShardRoutingCustomTest, TestFaceting, TermVectorComponentDistributedTest, HardAutoCommitTest]
   [junit4] Completed [244/536] on J2 in 1.78s, 1 test, 1 error <<< FAILURES!

[...truncated 787 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -XX:MaxPermSize=192m -ea -esa -Dtests.prefix=tests -Dtests.seed=E374FCA58FE83D09 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.4.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=5.4.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=ISO-8859-1 -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/test-framework/lib/junit4-ant-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/common/lucene-analyzers-common-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/lucene-codecs-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/lucene-backward-codecs-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/highlighter/lucene-highlighter-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/memory/lucene-memory-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/lucene-misc-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/lucene-spatial-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/expressions/lucene-expressions-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/suggest/lucene-suggest-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/grouping/lucene-grouping-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queries/lucene-queries-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/lucene-queryparser-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/lucene-join-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/lucene-sandbox-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/antlr4-runtime-4.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-commons-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-exec-1.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/concurrentlinkedhashmap-lru-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hppc-0.7.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/spatial4j-0.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-continuation-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-deploy-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-http-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-io-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-jmx-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-rewrite-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-security-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-server-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlet-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlets-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-util-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-webapp-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-xml-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr4-runtime-4.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150828_201611_790.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150828_201611_790.suites
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 210 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1485942732 bytes
Compression is 0.0%
Took 4 min 31 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 940 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/940/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=29013, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=29013, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50219/kx_: collection already exists: awholynewstresscollection_collection3_1
	at __randomizedtesting.SeedInfo.seed([E70F7E855E1CA177]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 10704 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/init-core-data-001
   [junit4]   2> 1597061 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[E70F7E855E1CA177]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1597061 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[E70F7E855E1CA177]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /kx_/
   [junit4]   2> 1597066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1597067 INFO  (Thread-21945) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1597067 INFO  (Thread-21945) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1597167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.ZkTestServer start zk server on port:57092
   [junit4]   2> 1597167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1597168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1597171 INFO  (zkCallback-1960-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3062a111 name:ZooKeeperConnection Watcher:127.0.0.1:57092 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1597171 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1597172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1597172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1597175 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14f7157f7e10000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1597175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1597176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1597178 INFO  (zkCallback-1961-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b9a88d5 name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1597178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1597178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1597178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1597181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1597182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1597184 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1597186 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1597187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1597190 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1597190 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1597193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1597194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1597196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1597197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1597199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1597199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1597202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1597202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1597204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1597204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1597207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1597207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1597209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1597210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1597212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1597212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1597215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1597215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1597218 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1597219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1597220 INFO  (zkCallback-1962-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7fbdb5a2 name:ZooKeeperConnection Watcher:127.0.0.1:57092 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1597220 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1597221 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1597221 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1597222 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14f7157f7e10002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1597223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1597223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1597225 INFO  (zkCallback-1963-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c9e8032 name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1597225 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1597225 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1597226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1597226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1597229 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1597229 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1597232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1597232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1597234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1597234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1597236 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1597237 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1597239 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1597239 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1597241 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1597241 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1597243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1597244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1597246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1597246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1597248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1597248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1597250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1597251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1597253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1597253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1597603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1
   [junit4]   2> 1597604 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1597606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1df6f665{/kx_,null,AVAILABLE}
   [junit4]   2> 1597607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.ServerConnector Started ServerConnector@75afdc91{HTTP/1.1}{127.0.0.1:48755}
   [junit4]   2> 1597607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.Server Started @1600810ms
   [junit4]   2> 1597607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/kx_, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/tempDir-001/control/data, hostPort=48755}
   [junit4]   2> 1597607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1597608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/'
   [junit4]   2> 1597624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1597624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1597627 INFO  (zkCallback-1964-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59c65308 name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1597627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1597627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1597629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1597629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/solr.xml
   [junit4]   2> 1597643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores
   [junit4]   2> 1597643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoreContainer New CoreContainer 1322089533
   [junit4]   2> 1597643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/]
   [junit4]   2> 1597643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/lib
   [junit4]   2> 1597643 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/lib).
   [junit4]   2> 1597652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1597653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1597653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1597654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1597654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57092/solr
   [junit4]   2> 1597654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1597654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1597655 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1597656 INFO  (zkCallback-1966-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b6d6638 name:ZooKeeperConnection Watcher:127.0.0.1:57092 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1597657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1597657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1597659 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1597660 INFO  (zkCallback-1967-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4fcf51fe name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1597660 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1597661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1597663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1597666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1597668 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1597670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1597673 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1597674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1597675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1597677 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1597678 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48755_kx_
   [junit4]   2> 1597678 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48755_kx_
   [junit4]   2> 1597680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1597681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1597683 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1597684 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94418739832881158-127.0.0.1:48755_kx_-n_0000000000
   [junit4]   2> 1597685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48755_kx_
   [junit4]   2> 1597685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1597686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer Overseer (id=94418739832881158-127.0.0.1:48755_kx_-n_0000000000) starting
   [junit4]   2> 1597689 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1597697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1597697 INFO  (OverseerCollectionConfigSetProcessor-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1597698 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1597698 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1597718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1597718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1597719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores
   [junit4]   2> 1597720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/, transient=false, schema=schema.xml, shard=, collection=control_collection, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1, loadOnStartup=true, name=collection1}
   [junit4]   2> 1597720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/
   [junit4]   2> 1597720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1597722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1597722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:48755_kx_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1597722 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1597722 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1597725 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:48755_kx_",
   [junit4]   2>   "base_url":"http://127.0.0.1:48755/kx_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 1597725 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1597726 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:48755_kx_",
   [junit4]   2>   "base_url":"http://127.0.0.1:48755/kx_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1597727 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1597728 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1597731 INFO  (zkCallback-1967-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1598727 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1598727 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1598728 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1598728 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1598729 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1598729 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/'
   [junit4]   2> 1598749 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1598758 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1598775 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1598807 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1598808 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1598816 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1599159 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1599168 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1599170 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1599180 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1599184 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1599188 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1599189 WARN  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1599189 WARN  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1599189 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1599190 WARN  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1599190 WARN  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1599192 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1599192 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1599192 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1599192 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1599193 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a95ed49
   [junit4]   2> 1599194 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/data
   [junit4]   2> 1599194 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1599194 WARN  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1599195 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/data/index
   [junit4]   2> 1599195 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=2, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4387319638306255]
   [junit4]   2> 1599195 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@30fa26da lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@770ccb54),segFN=segments_1,generation=1}
   [junit4]   2> 1599196 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1599196 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/data/
   [junit4]   2> 1599197 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1599202 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1599203 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1599203 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1599203 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1599203 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1599204 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1599204 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1599204 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1599205 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1599209 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1599211 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1599212 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1599215 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1599220 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/schema,/update/json,/admin/file,/admin/luke,/admin/mbeans,/update/json/docs,/update,/admin/threads,standard,/admin/properties,/admin/plugins,/config,/get,/admin/system,/admin/segments,/replication,/admin/ping,/update/csv
   [junit4]   2> 1599222 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1599222 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1599222 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1599224 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1599224 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1599225 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=30, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4720859087271765]
   [junit4]   2> 1599225 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@30fa26da lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@770ccb54),segFN=segments_1,generation=1}
   [junit4]   2> 1599225 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1599225 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4f147248[collection1] main
   [junit4]   2> 1599226 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1599227 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1599227 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1599228 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1599228 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1599228 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1599228 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1599229 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1599229 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1599229 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1599230 INFO  (searcherExecutor-3107-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4f147248[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1599230 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1599230 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1599230 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1599230 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1510699839592071168
   [junit4]   2> 1599232 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1510699839592071168
   [junit4]   2> 1599233 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1599233 INFO  (coreLoadExecutor-3106-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1599233 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:48755/kx_ collection:control_collection shard:shard1
   [junit4]   2> 1599234 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1599238 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94418739832881158-core_node1-n_0000000000
   [junit4]   2> 1599239 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1599241 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1599241 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1599241 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48755/kx_/collection1/
   [junit4]   2> 1599241 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1599241 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1599241 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:48755/kx_/collection1/ has no replicas
   [junit4]   2> 1599241 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48755/kx_/collection1/ shard1
   [junit4]   2> 1599242 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1599242 INFO  (zkCallback-1967-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1599246 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:48755/kx_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1599349 INFO  (zkCallback-1967-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1599397 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:48755/kx_/collection1/ and leader is http://127.0.0.1:48755/kx_/collection1/
   [junit4]   2> 1599397 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48755/kx_
   [junit4]   2> 1599397 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1599397 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1599398 INFO  (coreZkRegister-3100-thread-1-processing-r:core_node1 n:127.0.0.1:48755_kx_ x:collection1 s:shard1 c:control_collection) [n:127.0.0.1:48755_kx_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1599399 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:48755_kx_",
   [junit4]   2>   "base_url":"http://127.0.0.1:48755/kx_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 1599400 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:48755_kx_",
   [junit4]   2>   "base_url":"http://127.0.0.1:48755/kx_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1599502 INFO  (zkCallback-1967-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1599724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1599724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1599726 INFO  (zkCallback-1969-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@685470e6 name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1599726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1599726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1599727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1599729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1599729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1599729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1599730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1599731 INFO  (zkCallback-1970-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@212eb538 name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1599731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1599731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1599733 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14f7157f7e10008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1599734 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 1599734 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1599836 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1600030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/cores/collection1
   [junit4]   2> 1600031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001
   [junit4]   2> 1600031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1600033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4fa16042{/kx_,null,AVAILABLE}
   [junit4]   2> 1600034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.ServerConnector Started ServerConnector@286004a2{HTTP/1.1}{127.0.0.1:39325}
   [junit4]   2> 1600034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.e.j.s.Server Started @1603237ms
   [junit4]   2> 1600034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/tempDir-001/jetty1, hostContext=/kx_, hostPort=39325, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/cores}
   [junit4]   2> 1600034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1600034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/'
   [junit4]   2> 1600050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1600051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1600052 INFO  (zkCallback-1971-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@214da534 name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1600053 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1600053 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1600054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1600054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/solr.xml
   [junit4]   2> 1600068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/cores
   [junit4]   2> 1600068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoreContainer New CoreContainer 1073505885
   [junit4]   2> 1600068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/]
   [junit4]   2> 1600068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/lib
   [junit4]   2> 1600069 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/lib).
   [junit4]   2> 1600077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1600078 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1600078 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1600079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1600079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57092/solr
   [junit4]   2> 1600079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1600079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1600079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1600081 INFO  (zkCallback-1973-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@47a397b1 name:ZooKeeperConnection Watcher:127.0.0.1:57092 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1600081 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1600082 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1600083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1600084 INFO  (zkCallback-1974-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76565f17 name:ZooKeeperConnection Watcher:127.0.0.1:57092/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1600084 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1600089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1601094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39325_kx_
   [junit4]   2> 1601095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39325_kx_
   [junit4]   2> 1601096 INFO  (zkCallback-1969-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1601096 INFO  (zkCallback-1974-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1601097 INFO  (zkCallback-1967-thread-1-processing-n:127.0.0.1:48755_kx_) [n:127.0.0.1:48755_kx_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1601099 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1601099 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94418739832881163-127.0.0.1:39325_kx_-n_0000000001
   [junit4]   2> 1601100 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94418739832881158-127.0.0.1:48755_kx_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1601114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1601115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1601116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/cores
   [junit4]   2> 1601117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, dataDir=data/, config=solrconfig.xml, configSetProperties=configsetprops.json, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/cores/collection1, collection=collection1, loadOnStartup=true, coreNodeName=, schema=schema.xml, name=collection1}
   [junit4]   2> 1601117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_E70F7E855E1CA177-001/shard-1-001/cores/collection1/
   [junit4]   2> 1601117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1601118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1601118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E70F7E855E1CA177]) [n:127.0.0.1:39325_kx_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1601118 INFO  (coreLoadExecutor-3117-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1601119 INFO  (coreLoadExecutor-3117-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1601120 INFO  (coreLoadExecutor-3117-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for external collection collection1
   [junit4]   2> 1601120 INFO  (coreLoadExecutor-3117-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1601120 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:39325_kx_",
   [junit4]   2>   "base_url":"http://127.0.0.1:39325/kx_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1"} current state version: 4
   [junit4]   2> 1601121 INFO  (coreLoadExecutor-3117-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1601121 INFO  (coreLoadExecutor-3117-thread-1-processing-n:127.0.0.1:39325_kx_) [n:127.0.0.1:39325_kx_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1601121 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:39325_kx_",
   [junit4]   2>   "base_url":"http://127.0.0.1:39325/kx_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1"}
   [junit4]   2> 1601121 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1601121 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1601224 INFO  (OverseerStateUpdate-94418739832881158-127.0.0.1:48755_kx_-n_0000000000) [n:127.0.0.1:48755_kx_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.js

[...truncated too long message...]

t]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39325/kx_: Could not find collection : awholynewstresscollection_collection4_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E70F7E855E1CA177]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=29015, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55594/kx_: collection already exists: awholynewstresscollection_collection5_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E70F7E855E1CA177]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=29011, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55594/kx_: collection already exists: awholynewstresscollection_collection1_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E70F7E855E1CA177]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]   2> 2028624 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[E70F7E855E1CA177]-worker) [n:127.0.0.1:36855_kx_ c:addReplicaColl s:shard2 r:core_node3 x:addReplicaColl_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {id=BlockTreeOrds(blocksize=128), multiDefault=PostingsFormat(name=LuceneFixedGap), _version_=FSTOrd50, text=PostingsFormat(name=SimpleText), range_facet_i_dv=FSTOrd50, a_t=FSTOrd50, other_tl1=FSTOrd50, range_facet_l_dv=BlockTreeOrds(blocksize=128), intDefault=FSTOrd50, a_i1=FSTOrd50, timestamp=FSTOrd50, range_facet_l=FSTOrd50}, docValues:{range_facet_i_dv=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_SD, timezone=MIT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=134712856,total=510656512
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolrCLIRunExample, SmileWriterTest, TestManagedStopFilterFactory, SignatureUpdateProcessorFactoryTest, StandardRequestHandlerTest, LeaderInitiatedRecoveryOnCommitTest, TestSolr4Spatial2, TestDocSet, TestSolrJ, TestConfigSets, TestStressLucene, LeaderElectionIntegrationTest, AlternateDirectoryTest, PreAnalyzedUpdateProcessorTest, TestComponentsName, TestFieldSortValues, TestValueSourceCache, BasicZkTest, VersionInfoTest, QueryElevationComponentTest, TestDocumentBuilder, MultiThreadedOCPTest, RequestLoggingTest, TestJettySolrRunner, AnalysisAfterCoreReloadTest, DistributedMLTComponentTest, TestDownShardTolerantSearch, DocumentAnalysisRequestHandlerTest, TestFieldTypeResource, SimpleCollectionCreateDeleteTest, DateFieldTest, ResponseLogComponentTest, ResourceLoaderTest, HdfsChaosMonkeySafeLeaderTest, CurrencyFieldOpenExchangeTest, JsonLoaderTest, TestDocBasedVersionConstraints, TestCryptoKeys, ReturnFieldsTest, UUIDUpdateProcessorFallbackTest, TestIBSimilarityFactory, DistributedSpellCheckComponentTest, FullHLLTest, CollectionsAPIAsyncDistributedZkTest, TestFreeTextSuggestions, TestOmitPositions, TestLuceneMatchVersion, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, TestHdfsUpdateLog, FileUtilsTest, TestFieldTypeCollectionResource, TestDistribIDF, TestBadConfig, RequiredFieldsTest, TestMiniSolrCloudClusterSSL, TestSolrQueryParserDefaultOperatorResource, SuggestComponentTest, IndexSchemaTest, DateMathParserTest, ExternalFileFieldSortTest, TestStressUserVersions, TestEmbeddedSolrServerConstructors, TestAnalyzedSuggestions, TestSchemaSimilarityResource, DistributedQueryElevationComponentTest, TestRandomDVFaceting, TestDynamicFieldResource, TestCloudManagedSchemaConcurrent, DeleteLastCustomShardedReplicaTest, TestFunctionQuery, DocValuesMissingTest, TestCloudInspectUtil, TestRandomFaceting, XsltUpdateRequestHandlerTest, TestLMDirichletSimilarityFactory, TestFoldingMultitermQuery, TestCollationFieldDocValues, TestRealTimeGet, StressHdfsTest, TestMacros, TestSolr4Spatial, IndexSchemaRuntimeFieldTest, TestCustomDocTransformer, TimeZoneUtilsTest, PingRequestHandlerTest, DistributedFacetPivotLongTailTest, DistributedDebugComponentTest, BasicDistributedZkTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [275/536] on J0 in 432.32s, 1 test, 1 error <<< FAILURES!

[...truncated 859 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 536 suites, 2128 tests, 1 error, 29 ignored (17 assumptions)

Total time: 181 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1169049270 bytes
Compression is 0.0%
Took 3 min 21 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 939 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/939/

1 tests failed.
REGRESSION:  org.apache.lucene.codecs.lucene46.TestLucene46SegmentInfoFormat.testRandomExceptions

Error Message:
hit unexpected FileNotFoundException: file=_3g.si

Stack Trace:
java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3g.si
	at __randomizedtesting.SeedInfo.seed([1A10EDD0B2491A33:723F83142C474393]:0)
	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:753)
	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:530)
	at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:733)
	at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:4700)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4218)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 4862 lines...]
   [junit4] Suite: org.apache.lucene.codecs.lucene46.TestLucene46SegmentInfoFormat
   [junit4]   1> Unexpected exception: dumping fake-exception-log:...
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_8_Lucene41_0.doc)
   [junit4]   1> java.io.IOException: a random IOException (_8_Lucene41_0.doc)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOException(MockDirectoryWrapper.java:445)
   [junit4]   1> 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:151)
   [junit4]   1> 	at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:127)
   [junit4]   1> 	at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:70)
   [junit4]   1> 	at org.apache.lucene.codecs.CodecUtil.writeHeader(CodecUtil.java:91)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsWriter.<init>(Lucene41PostingsWriter.java:99)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsWriter.<init>(Lucene41PostingsWriter.java:160)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWPostingsFormat.fieldsConsumer(Lucene41RWPostingsFormat.java:40)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2813)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2977)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2944)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:468)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_c_Lucene41_0.doc)
   [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_c_Lucene41_0.doc)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.Directory.copyFrom(Directory.java:150)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundFormat.write(Lucene40CompoundFormat.java:51)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4680)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:492)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 	Suppressed: java.nio.file.NoSuchFileException: a random IOException (_c.cfe)
   [junit4]   1> 		at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 		at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 		at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 		at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 		at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 		at org.apache.lucene.codecs.lucene40.Lucene40CompoundWriter.close(Lucene40CompoundWriter.java:163)
   [junit4]   1> 		at org.apache.lucene.codecs.lucene40.Lucene40CompoundReader.close(Lucene40CompoundReader.java:160)
   [junit4]   1> 		at org.apache.lucene.codecs.lucene40.Lucene40CompoundFormat.write(Lucene40CompoundFormat.java:53)
   [junit4]   1> 		... 45 more
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_f_Lucene41_0.doc)
   [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_f_Lucene41_0.doc)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsWriter.<init>(Lucene41PostingsWriter.java:93)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsWriter.<init>(Lucene41PostingsWriter.java:160)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWPostingsFormat.fieldsConsumer(Lucene41RWPostingsFormat.java:40)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_h_1.del)
   [junit4]   1> java.io.IOException: a random IOException (_h_1.del)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.BitVector.write(BitVector.java:169)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40LiveDocsFormat.writeLiveDocs(Lucene40LiveDocsFormat.java:82)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:527)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_j.fnm)
   [junit4]   1> java.io.IOException: a random IOException (_j.fnm)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene46.Lucene46FieldInfosFormat.write(Lucene46FieldInfosFormat.java:133)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:118)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_v_Lucene41_0.tim)
   [junit4]   1> java.io.IOException: a random IOException (_v_Lucene41_0.tim)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.Directory.copyFrom(Directory.java:150)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundFormat.write(Lucene40CompoundFormat.java:51)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4680)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:492)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_x.si)
   [junit4]   1> java.io.IOException: a random IOException (_x.si)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene46.Lucene46RWSegmentInfoFormat.write(Lucene46RWSegmentInfoFormat.java:42)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:501)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_13.fnm)
   [junit4]   1> java.io.IOException: a random IOException (_13.fnm)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene46.Lucene46FieldInfosFormat.write(Lucene46FieldInfosFormat.java:133)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:118)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_15.fdx)
   [junit4]   1> java.io.IOException: a random IOException (_15.fdx)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41StoredFieldsReader.<init>(Lucene41StoredFieldsReader.java:131)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41StoredFieldsFormat.fieldsReader(Lucene41StoredFieldsFormat.java:44)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:117)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3170)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3156)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_16_Lucene41_0.doc)
   [junit4]   1> java.io.IOException: a random IOException (_16_Lucene41_0.doc)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsWriter.<init>(Lucene41PostingsWriter.java:93)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsWriter.<init>(Lucene41PostingsWriter.java:160)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWPostingsFormat.fieldsConsumer(Lucene41RWPostingsFormat.java:40)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1k_Lucene41_0.tip)
   [junit4]   1> java.io.IOException: a random IOException (_1k_Lucene41_0.tip)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsReader.<init>(Lucene40BlockTreeTermsReader.java:145)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:104)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:434)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1m_Lucene41_0.tim)
   [junit4]   1> java.io.IOException: a random IOException (_1m_Lucene41_0.tim)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsReader.<init>(Lucene40BlockTreeTermsReader.java:137)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:104)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:434)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1t_Lucene41_0.tip)
   [junit4]   1> java.io.IOException: a random IOException (_1t_Lucene41_0.tip)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsWriter.<init>(Lucene40BlockTreeTermsWriter.java:291)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWPostingsFormat.fieldsConsumer(Lucene41RWPostingsFormat.java:44)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_21_Lucene41_0.tip)
   [junit4]   1> java.io.IOException: a random IOException (_21_Lucene41_0.tip)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsWriter.<init>(Lucene40BlockTreeTermsWriter.java:291)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWPostingsFormat.fieldsConsumer(Lucene41RWPostingsFormat.java:44)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_25.fdx)
   [junit4]

[...truncated too long message...]

.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene46SegmentInfoFormat -Dtests.method=testRandomExceptions -Dtests.seed=1A10EDD0B2491A33 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=fr -Dtests.timezone=BET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.20s J2 | TestLucene46SegmentInfoFormat.testRandomExceptions <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3g.si
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1A10EDD0B2491A33:723F83142C474393]:0)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:753)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:530)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:733)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:4700)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4218)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
   [junit4]    > 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]    > 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
   [junit4]    > 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/J2/temp/lucene.codecs.lucene46.TestLucene46SegmentInfoFormat_1A10EDD0B2491A33-001
   [junit4]   2> NOTE: test params are: codec=Lucene53, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=fr, timezone=BET
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=58932968,total=407371776
   [junit4]   2> NOTE: All tests run in this JVM: [TestLucene49DocValuesFormat, TestLucene40CompoundFormat, TestLucene41ForUtil, TestLucene45DocValuesFormat, TestDocValuesUpdatesOnOldSegments, TestBackwardsCompatibility, TestReuseDocsEnum, TestLucene40BlockFormat, TestLucene40SegmentInfoFormat, TestLucene40StoredFieldsFormat, TestLucene42FieldInfoFormat, TestLucene42NormsFormat, TestLucene46FieldInfoFormat, TestLucene46SegmentInfoFormat]
   [junit4] Completed [27/30] on J2 in 0.36s, 15 tests, 1 failure <<< FAILURES!

[...truncated 27 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 30 suites, 806 tests, 1 failure, 81 ignored (81 assumptions)

Total time: 83 minutes 38 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1711049 bytes
Compression is 0.0%
Took 0.44 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 938 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/938/

3 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Wed Aug 26 10:23:08 MIST 2015

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Wed Aug 26 10:23:08 MIST 2015
	at __randomizedtesting.SeedInfo.seed([6663C668C56EB0BE:BDC8C6AEC046D90D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1417)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:769)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:42181: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42181: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=62308, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=62308, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40454/n/ai: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([6663C668C56EB0BE]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10482 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/init-core-data-001
   [junit4]   2> 363097 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6663C668C56EB0BE]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 363097 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6663C668C56EB0BE]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 363102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 363102 INFO  (Thread-1293) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 363103 INFO  (Thread-1293) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 363202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.ZkTestServer start zk server on port:53004
   [junit4]   2> 363203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 363203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 363206 INFO  (zkCallback-221-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3f3cad5e name:ZooKeeperConnection Watcher:127.0.0.1:53004 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 363206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 363206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 363206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 363209 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14f67112b950000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 363209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 363210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 363212 INFO  (zkCallback-222-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bb6c738 name:ZooKeeperConnection Watcher:127.0.0.1:53004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 363212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 363212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 363212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 363214 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 363216 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 363218 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 363220 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 363220 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 363223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 363223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 363225 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 363226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 363228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 363228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 363230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 363230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 363232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 363233 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 363234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 363235 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 363236 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 363237 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 363238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 363239 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 363240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 363241 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 363243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 363243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 363523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1
   [junit4]   2> 363525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 363527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60ac3905{/,null,AVAILABLE}
   [junit4]   2> 363530 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4ad3a8d8{HTTP/1.1}{127.0.0.1:39900}
   [junit4]   2> 363530 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.Server Started @367032ms
   [junit4]   2> 363530 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=39900, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores, hostContext=/}
   [junit4]   2> 363530 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 363530 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/'
   [junit4]   2> 363547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 363550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 363553 INFO  (zkCallback-223-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ab2ecd3 name:ZooKeeperConnection Watcher:127.0.0.1:53004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 363553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 363553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 363555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 363555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/solr.xml
   [junit4]   2> 363570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores
   [junit4]   2> 363570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoreContainer New CoreContainer 1521765746
   [junit4]   2> 363570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/]
   [junit4]   2> 363570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/lib
   [junit4]   2> 363570 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/lib).
   [junit4]   2> 363578 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 363580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 363580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 363580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 363581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53004/solr
   [junit4]   2> 363581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 363581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 363581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 363583 INFO  (zkCallback-225-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f7244bb name:ZooKeeperConnection Watcher:127.0.0.1:53004 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 363583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 363584 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 363585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 363586 INFO  (zkCallback-226-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@425c045e name:ZooKeeperConnection Watcher:127.0.0.1:53004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 363587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 363588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 363590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 363592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 363594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 363596 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 363599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 363600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 363602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 363603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 363604 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39900_
   [junit4]   2> 363605 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39900_
   [junit4]   2> 363607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 363608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 363610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 363611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94407440641359876-127.0.0.1:39900_-n_0000000000
   [junit4]   2> 363611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39900_
   [junit4]   2> 363612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 363613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer Overseer (id=94407440641359876-127.0.0.1:39900_-n_0000000000) starting
   [junit4]   2> 363615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 363623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 363623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 363623 INFO  (OverseerCollectionProcessor-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.OverseerProcessor Process current queue of overseer operations
   [junit4]   2> 363625 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 363641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 363641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 363643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores
   [junit4]   2> 363644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/, coreNodeName=, shard=, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, collection=control_collection, loadOnStartup=true, name=collection1, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1, schema=schema.xml}
   [junit4]   2> 363644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/
   [junit4]   2> 363644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 363645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 363646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:39900_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 363646 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 363646 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 363648 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:39900_",
   [junit4]   2>   "base_url":"http://127.0.0.1:39900",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 363648 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 363649 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:39900_",
   [junit4]   2>   "base_url":"http://127.0.0.1:39900",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null}
   [junit4]   2> 363649 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 363650 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 363652 INFO  (zkCallback-226-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 364649 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 364649 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 364650 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 364650 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 364651 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 364651 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/'
   [junit4]   2> 364667 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 364674 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 364687 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 364716 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 364718 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 364732 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 364993 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 365002 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 365005 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 365016 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 365020 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 365022 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 365023 WARN  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 365023 WARN  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 365024 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 365024 WARN  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 365025 WARN  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 365026 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 365026 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 365026 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 365026 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 365026 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@460dab4d
   [junit4]   2> 365027 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/data
   [junit4]   2> 365028 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/data/index/
   [junit4]   2> 365028 WARN  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 365028 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/data/index
   [junit4]   2> 365029 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=758179315, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 365029 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4a423551 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@40ba7eed),segFN=segments_1,generation=1}
   [junit4]   2> 365030 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 365030 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/data/
   [junit4]   2> 365030 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 365034 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 365034 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 365035 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 365035 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 365035 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 365035 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 365035 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 365035 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 365036 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 365038 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 365040 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 365041 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 365042 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 365046 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/luke,/admin/system,/admin/file,/admin/properties,/admin/plugins,standard,/update/csv,/get,/update,/config,/admin/mbeans,/replication,/update/json/docs,/admin/segments,/schema,/update/json,/admin/threads,/admin/ping,/admin/logging
   [junit4]   2> 365047 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 365048 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 365048 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 365049 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 365049 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 365050 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 365050 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4a423551 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@40ba7eed),segFN=segments_1,generation=1}
   [junit4]   2> 365050 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 365050 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@18c1d69b[collection1] main
   [junit4]   2> 365051 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 365052 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 365052 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 365052 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 365052 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 365052 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 365053 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 365053 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 365053 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 365053 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 365054 INFO  (searcherExecutor-873-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@18c1d69b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 365054 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 365054 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 365054 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 365054 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1510519052301238272
   [junit4]   2> 365056 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1510519052301238272
   [junit4]   2> 365057 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 365057 INFO  (coreLoadExecutor-872-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 365058 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:39900 collection:control_collection shard:shard1
   [junit4]   2> 365059 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 365062 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94407440641359876-core_node1-n_0000000000
   [junit4]   2> 365063 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 365065 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 365066 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 365066 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 365066 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39900/collection1/
   [junit4]   2> 365066 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 365075 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39900/collection1/ has no replicas
   [junit4]   2> 365076 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39900/collection1/ shard1
   [junit4]   2> 365076 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 365081 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39900",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 365148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 365148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 365151 INFO  (zkCallback-228-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56b6a996 name:ZooKeeperConnection Watcher:127.0.0.1:53004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 365151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 365152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 365152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 365155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 365184 INFO  (zkCallback-228-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 365184 INFO  (zkCallback-226-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 365232 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:39900/collection1/ and leader is http://127.0.0.1:39900/collection1/
   [junit4]   2> 365232 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39900
   [junit4]   2> 365232 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 365233 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 365233 INFO  (coreZkRegister-866-thread-1-processing-c:control_collection n:127.0.0.1:39900_ x:collection1 r:core_node1 s:shard1) [n:127.0.0.1:39900_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 365235 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:39900_",
   [junit4]   2>   "base_url":"http://127.0.0.1:39900",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1"} current state version: 2
   [junit4]   2> 365236 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:39900_",
   [junit4]   2>   "base_url":"http://127.0.0.1:39900",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 365339 INFO  (zkCallback-226-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 365339 INFO  (zkCallback-228-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 365449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores/collection1
   [junit4]   2> 365450 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001
   [junit4]   2> 365451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 365453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f507799{/,null,AVAILABLE}
   [junit4]   2> 365453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@307e6524{HTTP/1.1}{127.0.0.1:42181}
   [junit4]   2> 365453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.e.j.s.Server Started @368955ms
   [junit4]   2> 365453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores, solrconfig=solrconfig.xml, hostPort=42181, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/tempDir-001/jetty1}
   [junit4]   2> 365454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 365454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/'
   [junit4]   2> 365470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 365471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 365472 INFO  (zkCallback-229-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ca32981 name:ZooKeeperConnection Watcher:127.0.0.1:53004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 365472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 365473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 365474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 365474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/solr.xml
   [junit4]   2> 365489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores
   [junit4]   2> 365489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoreContainer New CoreContainer 936293789
   [junit4]   2> 365489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/]
   [junit4]   2> 365489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/lib
   [junit4]   2> 365489 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/lib).
   [junit4]   2> 365497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 365499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 365500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 365500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 365500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53004/solr
   [junit4]   2> 365500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 365500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 365501 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 365502 INFO  (zkCallback-231-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d65d824 name:ZooKeeperConnection Watcher:127.0.0.1:53004 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 365503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 365503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 365505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 365507 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@86e3392 name:ZooKeeperConnection Watcher:127.0.0.1:53004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 365507 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 365511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 366516 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42181_
   [junit4]   2> 366517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:42181_
   [junit4]   2> 366518 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 366518 INFO  (zkCallback-228-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 366519 INFO  (zkCallback-226-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 366520 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 366521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94407440641359880-127.0.0.1:42181_-n_0000000001
   [junit4]   2> 366522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94407440641359876-127.0.0.1:39900_-n_0000000000 to know if I could be the leader
   [junit4]   2> 366532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 366532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 366533 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores
   [junit4]   2> 366534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores/collection1, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores/collection1/, name=collection1, collection=collection1, transient=false, schema=schema.xml, dataDir=data/, loadOnStartup=true, config=solrconfig.xml, shard=, coreNodeName=}
   [junit4]   2> 366534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores/collection1/
   [junit4]   2> 366534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 366535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 366536 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6663C668C56EB0BE]) [n:127.0.0.1:42181_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 366536 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 366536 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 366538 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:42181_",
   [junit4]   2>   "base_url":"http://127.0.0.1:42181",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null} current state version: 3
   [junit4]   2> 366538 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 366538 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:42181_",
   [junit4]   2>   "base_url":"http://127.0.0.1:42181",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null}
   [junit4]   2> 366538 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 366539 INFO  (OverseerStateUpdate-94407440641359876-127.0.0.1:39900_-n_0000000000) [n:127.0.0.1:39900_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 366642 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 366642 INFO  (zkCallback-228-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 366642 INFO  (zkCallback-226-thread-1-processing-n:127.0.0.1:39900_) [n:127.0.0.1:39900_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 367539 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 367539 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 367540 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 367540 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 367541 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 367541 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6663C668C56EB0BE-001/shard-1-001/cores/collection1/'
   [junit4]   2> 367559 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n:127.0.0.1:42181_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 367569 INFO  (coreLoadExecutor-883-thread-1-processing-n:127.0.0.1:42181_) [n

[...truncated too long message...]

eckMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3726157 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6663C668C56EB0BE]-worker) [n:127.0.0.1:42279_n%2Fai c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 25, 2015 7:49:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=sr_BA, timezone=SystemV/CST6CDT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=269762608,total=532676608
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldSortValues, JavabinLoaderTest, TestRealTimeGet, ShardSplitTest, TestCloudInspectUtil, TestCollectionAPI, SpatialRPTFieldTypeTest, ExternalFileFieldSortTest, URLClassifyProcessorTest, AsyncCallRequestStatusResponseTest, MinimalSchemaTest, QueryElevationComponentTest, UniqFieldsUpdateProcessorFactoryTest, RequestHandlersTest, UpdateRequestProcessorFactoryTest, TestSolrIndexConfig, DebugComponentTest, TestIntervalFaceting, TestMiniSolrCloudClusterSSL, IndexBasedSpellCheckerTest, TestAnalyzeInfixSuggestions, BadCopyFieldTest, CollectionsAPIAsyncDistributedZkTest, TestRandomRequestDistribution, TestQuerySenderListener, TestHdfsUpdateLog, TestRebalanceLeaders, TestPhraseSuggestions, TestCSVLoader, TestLazyCores, SolrCoreTest, DistributedFacetPivotLongTailTest, TermVectorComponentTest, SuggesterWFSTTest, DateFieldTest, TestFieldTypeResource, TestMergePolicyConfig, DefaultValueUpdateProcessorTest, TestWordDelimiterFilterFactory, SchemaVersionSpecificBehaviorTest, HdfsRecoverLeaseTest, IndexSchemaTest, TestBulkSchemaConcurrent, TestFaceting, TestJmxMonitoredMap, HdfsBasicDistributedZk2Test, TestRandomFaceting, HdfsDirectoryTest, HdfsUnloadDistributedZkTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SimpleCollectionCreateDeleteTest, TestManagedSchema, ConcurrentDeleteAndCreateCollectionTest, CachingDirectoryFactoryTest, PingRequestHandlerTest, TestExactStatsCache, TestReplicationHandler, AddSchemaFieldsUpdateProcessorFactoryTest, TestReversedWildcardFilterFactory, TimeZoneUtilsTest, SimplePostToolTest, UUIDUpdateProcessorFallbackTest, CoreMergeIndexesAdminHandlerTest, SmileWriterTest, TestDynamicFieldResource, TestRestoreCore, HdfsChaosMonkeySafeLeaderTest, TestConfigSets, SegmentsInfoRequestHandlerTest, TestSolrDynamicMBean, DeleteShardTest, TestJettySolrRunner, HighlighterConfigTest, TestFunctionQuery, DistribCursorPagingTest, TestSolrCLIRunExample, ClusterStateTest, TestCodecSupport, PeerSyncTest, BufferStoreTest, TestSolr4Spatial, BitVectorTest, StressHdfsTest, TestJmxIntegration, TermsComponentTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestTolerantSearch, ActionThrottleTest, AliasIntegrationTest, CleanupOldIndexTest, CollectionTooManyReplicasTest, DeleteInactiveReplicaTest, DeleteLastCustomShardedReplicaTest, DistribJoinFromCollectionTest, LeaderFailoverAfterPartitionTest, LeaderInitiatedRecoveryOnCommitTest, MigrateRouteKeyTest, OverseerCollectionQueueTest, OverseerRolesTest, RemoteQueryErrorTest, SharedFSAutoReplicaFailoverUtilsTest, SolrCloudExampleTest, SolrXmlInZkTest, TestCloudPivotFacet, TestDownShardTolerantSearch, TestLeaderElectionZkExpiry, TestRequestStatusCollectionAPI, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [451/534] on J0 in 390.32s, 1 test, 1 error <<< FAILURES!

[...truncated 261 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 534 suites, 2119 tests, 2 errors, 1 failure, 26 ignored (14 assumptions)

Total time: 184 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1137962082 bytes
Compression is 0.0%
Took 3 min 7 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 937 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/937/

4 tests failed.
REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([A011256E9FB0E14A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([A011256E9FB0E14A]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=68286, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=68286, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:40041/x_/a, https://127.0.0.1:37706/x_/a, https://127.0.0.1:54899/x_/a, https://127.0.0.1:43188/x_/a, https://127.0.0.1:44578/x_/a]
	at __randomizedtesting.SeedInfo.seed([A011256E9FB0E14A]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:40041/x_/a, https://127.0.0.1:37706/x_/a, https://127.0.0.1:54899/x_/a, https://127.0.0.1:43188/x_/a, https://127.0.0.1:44578/x_/a]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:43188/x_/a
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:570)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 9 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=26511, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=26511, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58176: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([A011256E9FB0E14A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 11012 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/init-core-data-001
   [junit4]   2> 1843877 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A011256E9FB0E14A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 1843877 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A011256E9FB0E14A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /x_/a
   [junit4]   2> 1843881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1843882 INFO  (Thread-62299) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1843882 INFO  (Thread-62299) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1843982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.ZkTestServer start zk server on port:53829
   [junit4]   2> 1843982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1843994 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1844019 INFO  (zkCallback-1210-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ccf0678 name:ZooKeeperConnection Watcher:127.0.0.1:53829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1844019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1844020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1844020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1844022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1844035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1844050 INFO  (zkCallback-1211-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f0fbc4f name:ZooKeeperConnection Watcher:127.0.0.1:53829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1844050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1844050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1844051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1844052 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1844053 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1844054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1844056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1844056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1844058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1844058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1844060 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1844060 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1844061 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1844061 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1844063 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1844063 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1844064 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1844064 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1844066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1844066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1844067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1844067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1844069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1844069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1844070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1844070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1844072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1844072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1844074 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1844089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1844095 INFO  (zkCallback-1212-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2020f4fe name:ZooKeeperConnection Watcher:127.0.0.1:53829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1844095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1844095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1844726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1
   [junit4]   2> 1844728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1844832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@531a285{/x_/a,null,AVAILABLE}
   [junit4]   2> 1844838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1966bae{SSL-http/1.1}{127.0.0.1:37706}
   [junit4]   2> 1844838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.Server Started @1847743ms
   [junit4]   2> 1844838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=37706, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores, hostContext=/x_/a, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/tempDir-001/control/data}
   [junit4]   2> 1844839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1844839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/'
   [junit4]   2> 1844854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1844856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1844856 INFO  (zkCallback-1213-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5cbd4561 name:ZooKeeperConnection Watcher:127.0.0.1:53829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1844857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1844857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1844858 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1844858 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/solr.xml
   [junit4]   2> 1844872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores
   [junit4]   2> 1844873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1950507020
   [junit4]   2> 1844873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/]
   [junit4]   2> 1844873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/lib
   [junit4]   2> 1844873 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/lib).
   [junit4]   2> 1844881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1844883 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1844885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1844885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1844885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53829/solr
   [junit4]   2> 1844885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1844886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1844887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1844888 INFO  (zkCallback-1215-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72d51d88 name:ZooKeeperConnection Watcher:127.0.0.1:53829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1844888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1844888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1844902 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1844913 INFO  (zkCallback-1216-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49bfbc10 name:ZooKeeperConnection Watcher:127.0.0.1:53829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1844914 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1844916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1844918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1844920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1844921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1844923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1844924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1844925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1844938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1844940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1844940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37706_x_%2Fa
   [junit4]   2> 1844941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37706_x_%2Fa
   [junit4]   2> 1844942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1844943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1844944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1844944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000
   [junit4]   2> 1844945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37706_x_%2Fa
   [junit4]   2> 1844945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1844946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer Overseer (id=94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) starting
   [junit4]   2> 1844947 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1844952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1844953 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1844953 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1844957 INFO  (OverseerCollectionProcessor-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.OverseerProcessor Process current queue of overseer operations
   [junit4]   2> 1844966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1844978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1844978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1844979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores
   [junit4]   2> 1844980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, shard=, schema=schema.xml, config=solrconfig.xml, dataDir=data/, loadOnStartup=true, name=collection1, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1, transient=false, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/}
   [junit4]   2> 1844980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/
   [junit4]   2> 1844980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1844998 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1844999 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1845006 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1845015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1845015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1845015 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"https://127.0.0.1:37706/x_/a",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:37706_x_%2Fa",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1845016 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"https://127.0.0.1:37706/x_/a",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:37706_x_%2Fa",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1845016 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1845016 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1845018 INFO  (zkCallback-1216-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1845018 INFO  (zkCallback-1216-thread-2-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1846006 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1846006 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1846007 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1846008 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1846008 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1846008 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/'
   [junit4]   2> 1846039 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1846052 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1846065 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1846128 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1846129 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1846138 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1846599 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1846608 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1846609 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1846619 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1846622 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1846625 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1846625 WARN  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1846625 WARN  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1846626 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1846626 WARN  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1846626 WARN  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1846627 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1846628 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1846628 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1846628 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1846628 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@42abbd83
   [junit4]   2> 1846629 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/data
   [junit4]   2> 1846629 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1846629 WARN  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1846630 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/data/index
   [junit4]   2> 1846630 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1846631 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2cdae0c7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@55db3fd2),segFN=segments_1,generation=1}
   [junit4]   2> 1846631 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1846634 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1846635 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1846635 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1846635 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1846635 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1846635 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1846636 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1846636 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1846636 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1846639 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1846640 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/data/
   [junit4]   2> 1846640 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1846640 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1846642 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1846643 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1846646 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/admin/mbeans,/admin/system,/admin/luke,standard,/admin/segments,/get,/admin/ping,/admin/plugins,/admin/threads,/update/json/docs,/replication,/config,/update/json,/update/csv,/admin/file,/admin/properties,/schema,/update
   [junit4]   2> 1846648 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1846651 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1846652 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1846652 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1846652 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1846653 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=623627695, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1846654 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2cdae0c7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@55db3fd2),segFN=segments_1,generation=1}
   [junit4]   2> 1846654 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1846654 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2d5cf74f[collection1] main
   [junit4]   2> 1846654 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1846675 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1846675 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1846676 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1846676 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1846676 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1846676 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1846677 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1846677 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1846677 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1846678 INFO  (searcherExecutor-3890-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2d5cf74f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1846678 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1846679 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1846679 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1846680 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1510431147344003072
   [junit4]   2> 1846684 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 5.0ms to seed version buckets with highest version 1510431147344003072
   [junit4]   2> 1846684 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1846690 INFO  (coreLoadExecutor-3889-thread-1-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1846694 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:37706/x_/a collection:control_collection shard:shard1
   [junit4]   2> 1846706 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1846716 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94401946525696005-core_node1-n_0000000000
   [junit4]   2> 1846717 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1846719 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1846719 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1846719 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37706/x_/a/collection1/
   [junit4]   2> 1846719 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1846719 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:37706/x_/a/collection1/ has no replicas
   [junit4]   2> 1846719 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37706/x_/a/collection1/ shard1
   [junit4]   2> 1846719 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1846722 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1846731 INFO  (zkCallback-1216-thread-3-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1846731 INFO  (zkCallback-1216-thread-2-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1846735 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:37706/x_/a",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1846837 INFO  (zkCallback-1216-thread-3-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1846837 INFO  (zkCallback-1216-thread-2-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1846883 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:37706/x_/a/collection1/ and leader is https://127.0.0.1:37706/x_/a/collection1/
   [junit4]   2> 1846883 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:37706/x_/a
   [junit4]   2> 1846884 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1846884 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1846884 INFO  (coreZkRegister-3883-thread-1-processing-s:shard1 c:control_collection x:collection1 n:127.0.0.1:37706_x_%2Fa r:core_node1) [n:127.0.0.1:37706_x_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1846886 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"https://127.0.0.1:37706/x_/a",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:37706_x_%2Fa",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1846886 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"https://127.0.0.1:37706/x_/a",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:37706_x_%2Fa",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1846989 INFO  (zkCallback-1216-thread-3-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1846989 INFO  (zkCallback-1216-thread-2-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1847093 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1847098 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1847122 INFO  (zkCallback-1218-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@693961fa name:ZooKeeperConnection Watcher:127.0.0.1:53829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1847129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1847130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1847130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1847135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1847473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores/collection1
   [junit4]   2> 1847474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001
   [junit4]   2> 1847475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1847482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c3cdf87{/x_/a,null,AVAILABLE}
   [junit4]   2> 1847485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@76ef0046{SSL-http/1.1}{127.0.0.1:43188}
   [junit4]   2> 1847485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.e.j.s.Server Started @1850390ms
   [junit4]   2> 1847485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=43188, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/x_/a, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores}
   [junit4]   2> 1847486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1847486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/'
   [junit4]   2> 1847524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1847527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1847559 INFO  (zkCallback-1219-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@536a2e04 name:ZooKeeperConnection Watcher:127.0.0.1:53829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1847560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1847560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1847561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1847561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/solr.xml
   [junit4]   2> 1847575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores
   [junit4]   2> 1847575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoreContainer New CoreContainer 307096892
   [junit4]   2> 1847576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/]
   [junit4]   2> 1847576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/lib
   [junit4]   2> 1847576 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/lib).
   [junit4]   2> 1847585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1847587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1847588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1847588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1847588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53829/solr
   [junit4]   2> 1847589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1847589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1847589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1847591 INFO  (zkCallback-1221-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1917aa9e name:ZooKeeperConnection Watcher:127.0.0.1:53829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1847591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1847591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1847594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1847595 INFO  (zkCallback-1222-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f304287 name:ZooKeeperConnection Watcher:127.0.0.1:53829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1847596 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1847613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1848617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43188_x_%2Fa
   [junit4]   2> 1848618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43188_x_%2Fa
   [junit4]   2> 1848619 INFO  (zkCallback-1216-thread-3-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1848619 INFO  (zkCallback-1218-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1848619 INFO  (zkCallback-1216-thread-2-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1848619 INFO  (zkCallback-1222-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1848621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1848622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94401946525696009-127.0.0.1:43188_x_%2Fa-n_0000000001
   [junit4]   2> 1848623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000 to know if I could be the leader
   [junit4]   2> 1848635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1848635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1848636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores
   [junit4]   2> 1848636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores/collection1, collection=collection1, schema=schema.xml, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores/collection1/, loadOnStartup=true, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 1848637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores/collection1/
   [junit4]   2> 1848637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1848638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1848638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A011256E9FB0E14A]) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1848639 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1848640 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1848641 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1848642 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"https://127.0.0.1:43188/x_/a",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:43188_x_%2Fa",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 1848642 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "base_url":"https://127.0.0.1:43188/x_/a",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:43188_x_%2Fa",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1848643 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1848643 INFO  (OverseerStateUpdate-94401946525696005-127.0.0.1:37706_x_%2Fa-n_0000000000) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1848644 INFO  (zkCallback-1218-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1848644 INFO  (zkCallback-1222-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1848644 INFO  (zkCallback-1216-thread-2-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1848644 INFO  (zkCallback-1216-thread-3-processing-n:127.0.0.1:37706_x_%2Fa) [n:127.0.0.1:37706_x_%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1849641 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1849641 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1849642 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1849642 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1849643 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1849643 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_A011256E9FB0E14A-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1849659 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1849666 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1849681 INFO  (coreLoadExecutor-3900-thread-1-processing-n:127.0.0.1:43188_x_%2Fa) [n:127.0.0.1:43188_x_%2Fa c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1849723 INFO  (coreLoadExecutor

[...truncated too long message...]

$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    2) Thread[id=197949, name=qtp1604647483-197949-selector-ServerConnectorManager@782b8d3a/0, state=RUNNABLE, group=TGRP-TestReplicationHandler]
   [junit4]   2>         at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]   2>         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
   [junit4]   2>         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
   [junit4]   2>         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
   [junit4]   2>         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=197966, name=Scheduler-1210607755, state=WAITING, group=TGRP-TestReplicationHandler]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    4) Thread[id=197950, name=qtp1604647483-197950-selector-ServerConnectorManager@782b8d3a/1, state=RUNNABLE, group=TGRP-TestReplicationHandler]
   [junit4]   2>         at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
   [junit4]   2>         at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
   [junit4]   2>         at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:193)
   [junit4]   2>         at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
   [junit4]   2>         at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
   [junit4]   2>         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:78)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
   [junit4]   2>         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
   [junit4]   2>         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
   [junit4]   2>         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {id=PostingsFormat(name=LuceneFixedGap), name=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=vi_VN, timezone=AGT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=22,free=260568480,total=498073600
   [junit4]   2> NOTE: All tests run in this JVM: [FullSolrCloudDistribCmdsTest, HighlighterMaxOffsetTest, DistributedDebugComponentTest, UnloadDistributedZkTest, TestPseudoReturnFields, DataDrivenBlockJoinTest, TestUtils, TestCollationField, TestConfigSetProperties, TestFreeTextSuggestions, QueryResultKeyTest, TestExceedMaxTermLength, LeaderElectionTest, PluginInfoTest, TestShardHandlerFactory, TestDefaultStatsCache, RecoveryZkTest, MoreLikeThisHandlerTest, TestClassNameShortening, TestBinaryResponseWriter, SimpleFacetsTest, TestPivotHelperCode, TestSolrConfigHandler, TestTrie, TestBulkSchemaAPI, TestManagedSynonymFilterFactory, HardAutoCommitTest, SolrCoreCheckLockOnStartupTest, TestMaxScoreQueryParser, AutoCommitTest, XsltUpdateRequestHandlerTest, TestFileDictionaryLookup, StatsComponentTest, HdfsRecoveryZkTest, ResponseLogComponentTest, OverseerTest, TestSolrQueryParserResource, TestRecovery, DirectUpdateHandlerTest, BinaryUpdateRequestHandlerTest, TestSearcherReuse, HLLUtilTest, SpellCheckCollatorTest, TestManagedSchemaDynamicFieldResource, FileBasedSpellCheckerTest, QueryParsingTest, SortByFunctionTest, CoreAdminCreateDiscoverTest, TestDefaultSimilarityFactory, TestCursorMarkWithoutUniqueKey, TestManagedStopFilterFactory, TestQuerySenderNoQuery, DistributedQueryComponentCustomSortTest, DistributedFacetPivotSmallAdvancedTest, TestSystemIdResolver, TestPartialUpdateDeduplication, SOLR749Test, SolrIndexSplitterTest, TestSortingResponseWriter, RecoveryAfterSoftCommitTest, TestCrossCoreJoin, TestDocSet, NotRequiredUniqueKeyTest, TestCSVResponseWriter, TestNonDefinedSimilarityFactory, TestRecoveryHdfs, PrimUtilsTest, PathHierarchyTokenizerFactoryTest, DOMUtilTest, TestDefaultSearchFieldResource, BadIndexSchemaTest, SolrTestCaseJ4Test, TestNoOpRegenerator, TestBM25SimilarityFactory, HttpPartitionTest, TestStressRecovery, OverseerCollectionQueueTest, HdfsBasicDistributedZkTest, BasicDistributedZk2Test, LoggingHandlerTest, CursorPagingTest, ConnectionManagerTest, ZkStateReaderTest, TestSort, SyncSliceTest, ShardRoutingTest, TestReplicationHandler]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandler -Dtests.seed=A011256E9FB0E14A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=vi_VN -Dtests.timezone=AGT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J2 | TestReplicationHandler (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A011256E9FB0E14A]:0)
   [junit4] Completed [534/534] on J2 in 7223.28s, 4 tests, 2 errors <<< FAILURES!

[...truncated 14 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 534 suites, 2108 tests, 1 suite-level error, 3 errors, 26 ignored (14 assumptions)

Total time: 266 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1163442500 bytes
Compression is 0.0%
Took 3 min 3 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 936 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/936/

3 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test

Error Message:
shard2 is not consistent.  Got 1190 from http://127.0.0.1:40588/djf/collection1lastClient and got 793 from http://127.0.0.1:52105/djf/collection1

Stack Trace:
java.lang.AssertionError: shard2 is not consistent.  Got 1190 from http://127.0.0.1:40588/djf/collection1lastClient and got 793 from http://127.0.0.1:52105/djf/collection1
	at __randomizedtesting.SeedInfo.seed([49D118F2C6A76DF4:C1852728685B000C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1246)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1225)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:165)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:56385: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56385: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=58711, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=58711, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50135/m/k: Could not find collection : awholynewstresscollection_collection5_1
	at __randomizedtesting.SeedInfo.seed([49D118F2C6A76DF4]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 9916 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/init-core-data-001
   [junit4]   2> 53286 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[49D118F2C6A76DF4]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 53286 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[49D118F2C6A76DF4]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 53303 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 53306 INFO  (Thread-221) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 53306 INFO  (Thread-221) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 53405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.ZkTestServer start zk server on port:45106
   [junit4]   2> 53422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 53463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 53499 INFO  (zkCallback-4-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@13d05b12 name:ZooKeeperConnection Watcher:127.0.0.1:45106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 53500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 53500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 53502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 53531 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14f5c96fd1c0000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 53532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 53540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 53543 INFO  (zkCallback-5-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c438065 name:ZooKeeperConnection Watcher:127.0.0.1:45106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 53543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 53544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 53544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 53549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 53552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 53555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 53560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 53561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 53573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 53573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 53577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 53577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 53581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 53581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 53584 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 53585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 53592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 53593 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 53596 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 53597 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 53600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 53600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 53603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 53604 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 53607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 53607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 53610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 53611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 53615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 53620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 53620 INFO  (zkCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25eb9d54 name:ZooKeeperConnection Watcher:127.0.0.1:45106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 53621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 53621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 53621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 53628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 53628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 53631 INFO  (zkCallback-7-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2296eb5f name:ZooKeeperConnection Watcher:127.0.0.1:45106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 53632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 53632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 53632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 53633 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 53637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 53637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 53641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 53642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 53649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 53649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 53654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 53654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 53657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 53657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 53662 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 53663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 53666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 53667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 53672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 53672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 53676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 53676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 53680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 53681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 53684 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 53685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 54047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1
   [junit4]   2> 54049 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 54051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19775e7c{/,null,AVAILABLE}
   [junit4]   2> 54054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@43c995d7{HTTP/1.1}{127.0.0.1:34839}
   [junit4]   2> 54054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.Server Started @56925ms
   [junit4]   2> 54054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores, hostPort=34839, hostContext=/}
   [junit4]   2> 54055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 54055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/'
   [junit4]   2> 54072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 54072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 54074 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2cd89cd3 name:ZooKeeperConnection Watcher:127.0.0.1:45106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 54074 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 54075 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 54077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 54078 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/solr.xml
   [junit4]   2> 54093 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores
   [junit4]   2> 54093 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoreContainer New CoreContainer 1602034660
   [junit4]   2> 54094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/]
   [junit4]   2> 54094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/lib
   [junit4]   2> 54094 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/lib).
   [junit4]   2> 54102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 54104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 54104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 54104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 54105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45106/solr
   [junit4]   2> 54116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 54117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 54117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 54119 INFO  (zkCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5eca840 name:ZooKeeperConnection Watcher:127.0.0.1:45106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 54120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 54120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 54127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 54129 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40cf50fd name:ZooKeeperConnection Watcher:127.0.0.1:45106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 54129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 54137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 54146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 54153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 54158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 54165 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 54178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 54182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 54185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 54187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 54189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34839_
   [junit4]   2> 54191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34839_
   [junit4]   2> 54196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 54198 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 54201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 54203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94395920759390214-127.0.0.1:34839_-n_0000000000
   [junit4]   2> 54208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34839_
   [junit4]   2> 54209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 54211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.Overseer Overseer (id=94395920759390214-127.0.0.1:34839_-n_0000000000) starting
   [junit4]   2> 54219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 54246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 54311 INFO  (OverseerCollectionProcessor-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.OverseerProcessor Process current queue of overseer operations
   [junit4]   2> 54311 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 54316 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 54452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 54452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 54454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores
   [junit4]   2> 54458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1, coreNodeName=, loadOnStartup=true, dataDir=data/, schema=schema.xml, shard=, configSetProperties=configsetprops.json, config=solrconfig.xml, transient=false, name=collection1}
   [junit4]   2> 54459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/
   [junit4]   2> 54459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 54466 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 54466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 54467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:34839_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 54466 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 54473 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 54475 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:34839_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34839",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 54486 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:34839_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34839",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null}
   [junit4]   2> 54488 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 54498 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 54503 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 55474 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 55474 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 55475 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 55475 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 55478 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 55479 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/'
   [junit4]   2> 55504 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 55516 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 55535 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 55593 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 55594 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 55604 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 55954 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 55966 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 55968 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 55994 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 56001 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 56004 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 56007 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 56007 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 56008 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 56009 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 56009 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 56012 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 56013 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 56013 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 56013 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 56013 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce9df7b
   [junit4]   2> 56019 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/data
   [junit4]   2> 56019 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/data/index/
   [junit4]   2> 56019 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 56019 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/data/index
   [junit4]   2> 56020 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12340862446468703]
   [junit4]   2> 56020 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2b0a6581 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@378e0e3a),segFN=segments_1,generation=1}
   [junit4]   2> 56021 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 56021 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/data/
   [junit4]   2> 56021 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 56029 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 56030 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 56030 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 56030 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 56030 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 56031 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 56031 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 56031 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 56031 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 56034 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 56038 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 56040 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 56042 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 56052 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /get,/admin/threads,/config,standard,/schema,/update/json,/update/json/docs,/update/csv,/admin/logging,/admin/segments,/admin/properties,/admin/ping,/replication,/admin/plugins,/admin/system,/admin/mbeans,/admin/luke,/admin/file,/update
   [junit4]   2> 56054 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 56055 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 56055 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 56056 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 56056 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 56057 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@201054b7
   [junit4]   2> 56057 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2b0a6581 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@378e0e3a),segFN=segments_1,generation=1}
   [junit4]   2> 56057 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 56058 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@70be3a8c[collection1] main
   [junit4]   2> 56058 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 56060 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 56060 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 56063 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 56063 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 56063 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 56064 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 56064 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 56064 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 56064 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 56067 INFO  (searcherExecutor-240-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@70be3a8c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 56067 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 56067 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 56067 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 56067 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1510334734984544256
   [junit4]   2> 56069 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1510334734984544256
   [junit4]   2> 56070 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 56072 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 56073 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34839 collection:control_collection shard:shard1
   [junit4]   2> 56080 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 56088 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94395920759390214-core_node1-n_0000000000
   [junit4]   2> 56090 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 56093 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 56093 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 56093 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 56094 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34839/collection1/
   [junit4]   2> 56095 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 56095 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34839/collection1/ has no replicas
   [junit4]   2> 56095 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34839/collection1/ shard1
   [junit4]   2> 56097 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 56097 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 56107 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:34839",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 56212 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 56259 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:34839/collection1/ and leader is http://127.0.0.1:34839/collection1/
   [junit4]   2> 56259 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34839
   [junit4]   2> 56259 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 56259 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 56259 INFO  (coreZkRegister-233-thread-1-processing-n:127.0.0.1:34839_ r:core_node1 c:control_collection x:collection1 s:shard1) [n:127.0.0.1:34839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 56263 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:34839_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34839",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1"} current state version: 3
   [junit4]   2> 56264 INFO  (OverseerStateUpdate-94395920759390214-127.0.0.1:34839_-n_0000000000) [n:127.0.0.1:34839_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:34839_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34839",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 56370 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 56484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 56484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 56486 INFO  (zkCallback-13-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b73133 name:ZooKeeperConnection Watcher:127.0.0.1:45106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 56486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 56487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 56487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 56493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 56822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/cores/collection1
   [junit4]   2> 56824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001
   [junit4]   2> 56825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 56829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@28b36fe{/,null,AVAILABLE}
   [junit4]   2> 56830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@482df9e9{HTTP/1.1}{127.0.0.1:56385}
   [junit4]   2> 56830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.e.j.s.Server Started @59701ms
   [junit4]   2> 56830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/cores, hostContext=/, hostPort=56385, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/tempDir-001/jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 56831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 56831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/'
   [junit4]   2> 56847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 56848 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 56850 INFO  (zkCallback-14-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@577af093 name:ZooKeeperConnection Watcher:127.0.0.1:45106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 56850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 56850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 56853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 56853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/solr.xml
   [junit4]   2> 56869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/cores
   [junit4]   2> 56869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoreContainer New CoreContainer 1250603471
   [junit4]   2> 56869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/]
   [junit4]   2> 56869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/lib
   [junit4]   2> 56870 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/lib).
   [junit4]   2> 56878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 56879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 56880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 56880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 56880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45106/solr
   [junit4]   2> 56880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 56881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 56881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 56883 INFO  (zkCallback-16-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10884d9a name:ZooKeeperConnection Watcher:127.0.0.1:45106 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 56883 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 56884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 56886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 56888 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:56385_) [n:127.0.0.1:56385_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@562ec4ca name:ZooKeeperConnection Watcher:127.0.0.1:45106/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 56888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 56896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 57904 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56385_
   [junit4]   2> 57905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56385_
   [junit4]   2> 57908 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:56385_) [n:127.0.0.1:56385_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 57908 INFO  (zkCallback-13-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 57908 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:34839_) [n:127.0.0.1:34839_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 57911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 57912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94395920759390218-127.0.0.1:56385_-n_0000000001
   [junit4]   2> 57915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94395920759390214-127.0.0.1:34839_-n_0000000000 to know if I could be the leader
   [junit4]   2> 57961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 57961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 57962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D118F2C6A76DF4]) [n:127.0.0.1:56385_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_49D118F2C6A76DF4-001/shard-1-001/cores
   [junit4]   2> 57963 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[49D1

[...truncated too long message...]

e/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150823_180306_887.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150823_180306_887.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 203 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1691283517 bytes
Compression is 0.0%
Took 4 min 33 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 935 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/935/

2 tests failed.
REGRESSION:  org.apache.lucene.index.Test4GBStoredFields.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([86DC7899EA2DC74D]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.Test4GBStoredFields

Error Message:
Suite timeout exceeded (>= 14400000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 14400000 msec).
	at __randomizedtesting.SeedInfo.seed([86DC7899EA2DC74D]:0)




Build Log:
[...truncated 1792 lines...]
   [junit4] Suite: org.apache.lucene.index.Test4GBStoredFields
   [junit4]   2> ago 22, 2015 11:14:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.index.Test4GBStoredFields
   [junit4]   2>    1) Thread[id=2825, name=SUITE-Test4GBStoredFields-seed#[86DC7899EA2DC74D], state=RUNNABLE, group=TGRP-Test4GBStoredFields]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:690)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:679)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2>    2) Thread[id=2826, name=TEST-Test4GBStoredFields.test-seed#[86DC7899EA2DC74D], state=RUNNABLE, group=TGRP-Test4GBStoredFields]
   [junit4]   2>         at java.lang.Thread.yield(Native Method)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:139)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:127)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.LZ4.encodeLiterals(LZ4.java:149)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.LZ4.encodeLastLiterals(LZ4.java:162)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.LZ4.compressHC(LZ4.java:544)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.CompressionMode$LZ4HighCompressor.compress(CompressionMode.java:180)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:235)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:165)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.finishStoredFields(DefaultIndexingChain.java:270)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:311)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1475)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.Test4GBStoredFields.test(Test4GBStoredFields.java:75)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2>    4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=Test4GBStoredFields -Dtests.method=test -Dtests.seed=86DC7899EA2DC74D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_BO -Dtests.timezone=America/Cayenne -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   14400s J1 | Test4GBStoredFields.test <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([86DC7899EA2DC74D]:0)
   [junit4]   2> ago 22, 2015 11:14:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> ago 22, 2015 11:15:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.index.Test4GBStoredFields: 
   [junit4]   2>    1) Thread[id=2826, name=TEST-Test4GBStoredFields.test-seed#[86DC7899EA2DC74D], state=RUNNABLE, group=TGRP-Test4GBStoredFields]
   [junit4]   2>         at org.apache.lucene.codecs.compressing.LZ4$HCHashTable.access$700(LZ4.java:275)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.LZ4.compressHC(LZ4.java:422)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.CompressionMode$LZ4HighCompressor.compress(CompressionMode.java:180)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:235)
   [junit4]   2>         at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:165)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.finishStoredFields(DefaultIndexingChain.java:270)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:311)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1475)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.Test4GBStoredFields.test(Test4GBStoredFields.java:75)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> ago 22, 2015 11:15:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=2826, name=TEST-Test4GBStoredFields.test-seed#[86DC7899EA2DC74D], state=RUNNABLE, group=TGRP-Test4GBStoredFields]
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.Test4GBStoredFields_86DC7899EA2DC74D-001
   [junit4]   2> Aug 22, 2015 10:15:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=2, maxDocsPerChunk=4, blockSize=647), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=2, blockSize=647)), sim=DefaultSimilarity, locale=es_BO, timezone=America/Cayenne
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=43813168,total=212336640
   [junit4]   2> NOTE: All tests run in this JVM: [TestBufferedChecksum, TestSimilarity2, TestIndexWriterForceMerge, TestSameTokenSamePosition, TestFuzzyQuery, TestQueryRescorer, TestSearch, TestDocIdSetBuilder, TestSimpleAttributeImpl, TestNot, TestTermVectorsWriter, Test2BTerms, TestFixedBitSet, TestMultiLevelSkipList, TestTrackingDirectoryWrapper, TestRAMDirectory, TestDirectoryReaderReopen, TestDateFilter, TestSimilarity, TestLogMergePolicy, TestAttributeSource, TestFlushByRamOrCountsPolicy, TestSubScorerFreqs, TestMatchAllDocsQuery, TestParallelReaderEmptyIndex, TestFastDecompressionMode, TestSloppyPhraseQuery2, TestTwoPhaseCommitTool, TestSortRandom, TestCrash, TestTotalHitCountCollector, TestCharTermAttributeImpl, TestIndexReaderClose, TestIndexWriterExceptions, TestNRTReaderWithThreads, TestIndexWriterFromReader, TestSpansEnum, TestFilteredQuery, TestRoaringDocIdSet, TestDocumentsWriterStallControl, TestDemo, TestSpanFirstQuery, TestDeterminizeLexicon, TestOfflineSorter, TestTragicIndexWriterDeadlock, TestSameScoresWithThreads, TestElevationComparator, TestDocumentWriter, TestByteSlices, TestNoDeletionPolicy, TestDemoParallelLeafReader, TestSpanOrQuery, TestTryDelete, TestPayloadBasics, TestWildcardRandom, TestSpanMultiTermQueryWrapper, TestSimilarityProvider, TestSegmentMerger, TestSizeBoundedForceMerge, TestBooleanMinShouldMatch, TestLockFactory, TestLRUFilterCache, TestBooleanQuery, Test4GBStoredFields]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=Test4GBStoredFields -Dtests.seed=86DC7899EA2DC74D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_BO -Dtests.timezone=America/Cayenne -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | Test4GBStoredFields (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 14400000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([86DC7899EA2DC74D]:0)
   [junit4] Completed [406/411] on J1 in 14420.20s, 1 test, 2 errors <<< FAILURES!

[...truncated 27 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 411 suites, 3324 tests, 1 suite-level error, 1 error, 24 ignored (20 assumptions)

Total time: 266 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 3519954 bytes
Compression is 0.0%
Took 0.68 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 934 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/934/

2 tests failed.
FAILED:  org.apache.lucene.codecs.lucene46.TestLucene46FieldInfoFormat.testRandomExceptions

Error Message:
hit unexpected FileNotFoundException: file=_5k.si

Stack Trace:
java.lang.AssertionError: hit unexpected FileNotFoundException: file=_5k.si
	at __randomizedtesting.SeedInfo.seed([DE5D0B60FB3DAD54:B67265A46533F4F4]:0)
	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:753)
	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:530)
	at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:456)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3680)
	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene46.TestLucene46SegmentInfoFormat.testRandomExceptions

Error Message:
hit unexpected FileNotFoundException: file=_5k.si

Stack Trace:
java.lang.AssertionError: hit unexpected FileNotFoundException: file=_5k.si
	at __randomizedtesting.SeedInfo.seed([DE5D0B60FB3DAD54:B67265A46533F4F4]:0)
	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:753)
	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:530)
	at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:456)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3680)
	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 4569 lines...]
   [junit4] Suite: org.apache.lucene.codecs.lucene46.TestLucene46SegmentInfoFormat
   [junit4]   1> Unexpected exception: dumping fake-exception-log:...
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_7_Lucene41_0.tim)
   [junit4]   1> java.io.IOException: a random IOException (_7_Lucene41_0.tim)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsWriter.<init>(Lucene40BlockTreeTermsWriter.java:279)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWPostingsFormat.fieldsConsumer(Lucene41RWPostingsFormat.java:44)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_a.fdx)
   [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_a.fdx)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41StoredFieldsReader.<init>(Lucene41StoredFieldsReader.java:131)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41StoredFieldsFormat.fieldsReader(Lucene41StoredFieldsFormat.java:44)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:117)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:434)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_f_1.del)
   [junit4]   1> java.io.IOException: a random IOException (_f_1.del)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.BitVector.<init>(BitVector.java:273)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40LiveDocsFormat.readLiveDocs(Lucene40LiveDocsFormat.java:62)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:73)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:434)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_k_Lucene41_0.tip)
   [junit4]   1> java.io.IOException: a random IOException (_k_Lucene41_0.tip)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsReader.<init>(Lucene40BlockTreeTermsReader.java:145)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:104)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:434)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_l.fdx)
   [junit4]   1> java.io.IOException: a random IOException (_l.fdx)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41StoredFieldsWriter.<init>(Lucene41StoredFieldsWriter.java:96)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWStoredFieldsFormat.fieldsWriter(Lucene41RWStoredFieldsFormat.java:35)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1475)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_l.cfs)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_l.cfs)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundWriter.getOutput(Lucene40CompoundWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundWriter.createOutput(Lucene40CompoundWriter.java:244)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundReader.createOutput(Lucene40CompoundReader.java:229)
   [junit4]   1> 	at org.apache.lucene.store.Directory.copyFrom(Directory.java:151)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundFormat.write(Lucene40CompoundFormat.java:51)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4680)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:492)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_m.cfs)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_m.cfs)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundReader.<init>(Lucene40CompoundReader.java:76)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundFormat.getCompoundReader(Lucene40CompoundFormat.java:43)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:93)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:434)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_u.si)
   [junit4]   1> java.io.IOException: a random IOException (_u.si)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene46.Lucene46RWSegmentInfoFormat.write(Lucene46RWSegmentInfoFormat.java:42)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:501)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_12.fdt)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_12.fdt)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41StoredFieldsWriter.<init>(Lucene41StoredFieldsWriter.java:99)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWStoredFieldsFormat.fieldsWriter(Lucene41RWStoredFieldsFormat.java:35)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1475)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_18.cfs)
   [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_18.cfs)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundWriter.getOutput(Lucene40CompoundWriter.java:107)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundWriter.createOutput(Lucene40CompoundWriter.java:244)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundReader.createOutput(Lucene40CompoundReader.java:229)
   [junit4]   1> 	at org.apache.lucene.store.Directory.copyFrom(Directory.java:151)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundFormat.write(Lucene40CompoundFormat.java:51)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4680)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:492)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1f.cfe)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_1f.cfe)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundWriter.close(Lucene40CompoundWriter.java:163)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundReader.close(Lucene40CompoundReader.java:160)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene40.Lucene40CompoundFormat.write(Lucene40CompoundFormat.java:53)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4680)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:492)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:459)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1i_Lucene41_0.tim)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_1i_Lucene41_0.tim)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsReader.<init>(Lucene40BlockTreeTermsReader.java:137)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:104)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3170)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3156)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1o_Lucene41_0.tip)
   [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_1o_Lucene41_0.tip)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.blocktree.Lucene40BlockTreeTermsReader.<init>(Lucene40BlockTreeTermsReader.java:145)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:104)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:434)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:459)
   [junit4]   1> 	at org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:48)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1s_Lucene41_0.doc)
   [junit4]   1> java.io.IOException: a random IOException (_1s_Lucene41_0.doc)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.<init>(Lucene41PostingsReader.java:74)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:97)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3170)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3156)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryRead

[...truncated too long message...]

  1> 	at org.apache.lucene.codecs.lucene41.Lucene41StoredFieldsWriter.<init>(Lucene41StoredFieldsWriter.java:105)
   [junit4]   1> 	at org.apache.lucene.codecs.lucene41.Lucene41RWStoredFieldsFormat.fieldsWriter(Lucene41RWStoredFieldsFormat.java:35)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1475)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene46FieldInfoFormat -Dtests.method=testRandomExceptions -Dtests.seed=DE5D0B60FB3DAD54 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sr_BA -Dtests.timezone=Asia/Tashkent -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.48s J0 | TestLucene46FieldInfoFormat.testRandomExceptions <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: hit unexpected FileNotFoundException: file=_5k.si
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DE5D0B60FB3DAD54:B67265A46533F4F4]:0)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:753)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:530)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:456)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3680)
   [junit4]    > 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]    > 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:429)
   [junit4]    > 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene46.TestLucene46FieldInfoFormat_DE5D0B60FB3DAD54-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=sr_BA, timezone=Asia/Tashkent
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=85800248,total=157810688
   [junit4]   2> NOTE: All tests run in this JVM: [TestLucene46UndeadNorms, TestLucene410DocValuesFormat, TestLucene46FieldInfoFormat]
   [junit4] Completed [26/30] on J0 in 0.77s, 11 tests, 1 failure <<< FAILURES!

[...truncated 30 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 30 suites, 806 tests, 2 failures, 81 ignored (81 assumptions)

Total time: 80 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 2032136 bytes
Compression is 0.0%
Took 0.68 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 933 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/933/

1 tests failed.
REGRESSION:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([262C49BF593E0E8:E1669F1E5E8824AF]:0)
	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
	at org.apache.lucene.util.fst.FST.addNode(FST.java:793)
	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
	at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
	at org.apache.lucene.util.fst.Builder.add(Builder.java:417)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:258)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.access$500(MemoryPostingsFormat.java:113)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:400)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4089)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:173)
	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)




Build Log:
[...truncated 706 lines...]
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDuelingCodecsAtNight -Dtests.method=testBigEquals -Dtests.seed=262C49BF593E0E8 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_SG -Dtests.timezone=Africa/Maseru -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   1696s J2 | TestDuelingCodecsAtNight.testBigEquals <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([262C49BF593E0E8:E1669F1E5E8824AF]:0)
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.addNode(FST.java:793)
   [junit4]    > 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.add(Builder.java:417)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:258)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.access$500(MemoryPostingsFormat.java:113)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:400)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
   [junit4]    > 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:173)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J2/temp/lucene.index.TestDuelingCodecsAtNight_262C49BF593E0E8-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {body=DFR I(F)LZ(0.3), titleTokenized=BM25(k1=1.2,b=0.75)}, locale=en_SG, timezone=Africa/Maseru
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=279261352,total=403701760
   [junit4]   2> NOTE: All tests run in this JVM: [TestOmitTf, TestSortedSetSortField, TestSegmentTermEnum, TestMultiTermConstantScore, TestMultiValuedNumericRangeQuery, TestIndexWriterMaxDocs, Test2BNumericDocValues, TestIsCurrent, TestDuelingCodecsAtNight]
   [junit4] Completed [128/411] on J2 in 1738.07s, 3 tests, 1 error <<< FAILURES!

[...truncated 920 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J2-20150819_234031_421.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid32523.hprof ...
   [junit4] Heap dump file created [421972600 bytes in 0.794 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 31 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1451: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1005: There were test failures: 411 suites, 3323 tests, 1 error, 25 ignored (21 assumptions)

Total time: 72 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 425486212 bytes
Compression is 0.0%
Took 1 min 48 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 932 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/932/

No tests ran.

Build Log:
[...truncated 160 lines...]
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x
org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNAuthenticationException: svn: E170001: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:62)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:771)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:189)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:141)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.createDefaultErrorMessage(HTTPRequest.java:455)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:191)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:480)
	... 35 more
ERROR: Subversion update failed
java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x at hudson.remoting.Channel@4e042042:lucene
	at hudson.FilePath.act(FilePath.java:916)
	at hudson.FilePath.act(FilePath.java:893)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:936)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:871)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:615)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:524)
	at hudson.model.Run.execute(Run.java:1706)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:211)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.scm.subversion.UpdaterException: failed to perform svn update
	... 14 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	... 13 more
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNAuthenticationException: svn: E170001: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:62)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:771)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:189)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:141)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.createDefaultErrorMessage(HTTPRequest.java:455)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:191)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:480)
	... 35 more
Retrying after 10 seconds
Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/.
Updating http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x at revision '2015-08-18T22:07:26.940 -0400'
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x
org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNAuthenticationException: svn: E170001: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:62)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:771)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:189)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:141)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.createDefaultErrorMessage(HTTPRequest.java:455)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:191)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:480)
	... 35 more
ERROR: Subversion update failed
java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x at hudson.remoting.Channel@4e042042:lucene
	at hudson.FilePath.act(FilePath.java:916)
	at hudson.FilePath.act(FilePath.java:893)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:936)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:871)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:615)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:524)
	at hudson.model.Run.execute(Run.java:1706)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:211)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.scm.subversion.UpdaterException: failed to perform svn update
	... 14 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	... 13 more
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNAuthenticationException: svn: E170001: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:62)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:771)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:189)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:141)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.createDefaultErrorMessage(HTTPRequest.java:455)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:191)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:480)
	... 35 more
Retrying after 10 seconds
Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/.
Updating http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x at revision '2015-08-18T22:07:26.940 -0400'
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x
org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNAuthenticationException: svn: E170001: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:62)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:771)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:189)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:141)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.createDefaultErrorMessage(HTTPRequest.java:455)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:191)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:480)
	... 35 more
ERROR: Subversion update failed
java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x at hudson.remoting.Channel@4e042042:lucene
	at hudson.FilePath.act(FilePath.java:916)
	at hudson.FilePath.act(FilePath.java:893)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:936)
	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:871)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:615)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:524)
	at hudson.model.Run.execute(Run.java:1706)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:211)
	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011)
	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.scm.subversion.UpdaterException: failed to perform svn update
	... 14 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
	at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
	at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
	at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
	at org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
	... 13 more
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/branches/branch_5x failed
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
	... 35 more
Caused by: org.tmatesoft.svn.core.SVNAuthenticationException: svn: E170001: OPTIONS request failed on '/repos/asf/lucene/dev/branches/branch_5x'
svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:62)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:771)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
	... 34 more
Caused by: svn: E170001: OPTIONS of '/repos/asf/lucene/dev/branches/branch_5x': 403 Forbidden (http://svn.apache.org)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:189)
	at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:141)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.createDefaultErrorMessage(HTTPRequest.java:455)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:191)
	at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:480)
	... 35 more
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 0.47 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 931 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/931/

3 tests failed.
REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Tue Aug 18 18:09:01 MEST 2015

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Tue Aug 18 18:09:01 MEST 2015
	at __randomizedtesting.SeedInfo.seed([E5F76943B73D63E3:3E5C6985B2150A50]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1417)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:769)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=18660, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=18660, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40450: collection already exists: awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([E5F76943B73D63E3]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=707, name=WRITER6, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=707, name=WRITER6, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([E5F76943B73D63E3]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1673)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1530)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:779)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1088)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:705)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
	... 21 more




Build Log:
[...truncated 10088 lines...]
   [junit4] Suite: org.apache.solr.search.TestReloadDeadlock
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001
   [junit4]   2> 109784 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 109785 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 109785 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 109785 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 109786 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 109786 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 109824 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 109842 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 109880 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 109880 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 109900 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 110175 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 110177 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 110179 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 110182 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 110206 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 110206 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 110207 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 110220 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 416217510
   [junit4]   2> 110220 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 110220 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
   [junit4]   2> 110221 WARN  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib).
   [junit4]   2> 110240 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 110242 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 110243 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 110243 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 110243 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 110243 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 110244 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001, shard=shard1, instanceDir=collection1, transient=false, schema=schema15.xml, loadOnStartup=true, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, collection=collection1, name=collection1, config=solrconfig-tlog.xml}
   [junit4]   2> 110252 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 110253 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 110253 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 110254 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 110281 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 110295 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 110342 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 110343 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 110365 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 110649 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 110651 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 110653 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 110661 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 110706 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 110707 INFO  (coreLoadExecutor-197-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 110707 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 110707 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 110708 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e3553f9
   [junit4]   2> 110709 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001
   [junit4]   2> 110709 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001/index/
   [junit4]   2> 110709 WARN  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 110709 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001/index
   [junit4]   2> 110710 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=40.779296875, floorSegmentMB=0.470703125, forceMergeDeletesPctAllowed=7.536259304488297, segmentsPerTier=43.0, maxCFSSegmentSizeMB=1.44140625, noCFSRatio=0.0
   [junit4]   2> 110712 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_1,generation=1}
   [junit4]   2> 110713 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 110715 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001/
   [junit4]   2> 110721 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 110722 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 110722 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 110722 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 110723 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 110723 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 110723 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 110723 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 110724 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 110727 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 110728 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 110730 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 110731 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 110737 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/admin/threads,/schema,/get,/admin/system,/admin/luke,/admin/file,/update/json/docs,/update/csv,/admin/properties,/admin/mbeans,standard,/admin/logging,/replication,/update/json,/admin/segments,/admin/plugins,/admin/ping,/config
   [junit4]   2> 110739 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 110740 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 110740 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 110741 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 110741 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 110741 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4377b92e
   [junit4]   2> 110742 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_1,generation=1}
   [junit4]   2> 110743 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 110744 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@26ca372a[collection1] main
   [junit4]   2> 110744 WARN  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 110744 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 110744 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 110745 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 110745 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 110746 INFO  (searcherExecutor-198-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@26ca372a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 110746 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 110746 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 110746 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 110747 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1509856283694465024
   [junit4]   2> 110749 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1509856283694465024
   [junit4]   2> 110749 INFO  (coreLoadExecutor-197-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 110751 INFO  (SUITE-TestReloadDeadlock-seed#[E5F76943B73D63E3]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 110757 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testReloadDeadlock
   [junit4]   2> 110764 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 110772 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@26079d81[collection1] realtime
   [junit4]   2> 110773 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=*:* (-9223372036854775807)} 0 14
   [junit4]   2> 110774 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 110774 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_2,generation=2}
   [junit4]   2> 110775 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 110775 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6b1974f6[collection1] main
   [junit4]   2> 110776 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 110776 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6b1974f6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 110777 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 3
   [junit4]   2> 110787 INFO  (WRITER1) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 110802 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (1)]} 0 17
   [junit4]   2> 110804 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (3)]} 0 16
   [junit4]   2> 110804 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[19 (2)]} 0 17
   [junit4]   2> 110820 INFO  (WRITER1) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_3,generation=3}
   [junit4]   2> 110820 INFO  (WRITER1) [    ] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 110826 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@44b22f93[collection1] main
   [junit4]   2> 110828 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@44b22f93[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):C1) Uninverting(_1(5.4.0):C1) Uninverting(_2(5.4.0):C1)))}
   [junit4]   2> 110830 INFO  (WRITER1) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 110833 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 43
   [junit4]   2> 110878 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4946346[collection1] realtime
   [junit4]   2> 110879 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-4&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-4)} 0 90
   [junit4]   2> 110880 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (6)]} 0 77
   [junit4]   2> 110880 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (7)]} 0 77
   [junit4]   2> 110881 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (5)]} 0 93
   [junit4]   2> 110896 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@416d55c[collection1] realtime
   [junit4]   2> 110896 INFO  (WRITER1) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 110898 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=8,id=10} DBQs=[DBQ{version=9,q=id:20}]
   [junit4]   2> 110909 INFO  (WRITER1) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_3,generation=3}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_4,generation=4}
   [junit4]   2> 110910 INFO  (WRITER1) [    ] o.a.s.c.SolrCore newest commit generation = 4
   [junit4]   2> 110955 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@176f61bb[collection1] realtime
   [junit4]   2> 110958 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-9&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:20 (-9)} 0 152
   [junit4]   2> 110958 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (13)]} 0 76
   [junit4]   2> 110960 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (8)]} 0 154
   [junit4]   2> 110961 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@32aa204d[collection1] main
   [junit4]   2> 110961 INFO  (WRITER1) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 110962 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@32aa204d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):C1) Uninverting(_1(5.4.0):C1) Uninverting(_4(5.4.0):C1) Uninverting(_7(5.4.0):C1) Uninverting(_6(5.4.0):C1) Uninverting(_5(5.4.0):C1)))}
   [junit4]   2> 110964 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (15)]} 0 0
   [junit4]   2> 110964 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 121
   [junit4]   2> 110967 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (12)]} 0 86
   [junit4]   2> 110967 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (14)]} 0 8
   [junit4]   2> 110967 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (10)]} 0 87
   [junit4]   2> 110968 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[21 (19)]} 0 0
   [junit4]   2> 110968 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (11)]} 0 87
   [junit4]   2> 110968 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (18)]} 0 1
   [junit4]   2> 110969 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (21)]} 0 0
   [junit4]   2> 110969 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[21 (22)]} 0 0
   [junit4]   2> 110968 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (20)]} 0 0
   [junit4]   2> 110969 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (17)]} 0 4
   [junit4]   2> 110974 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (16)]} 0 13
   [junit4]   2> 110982 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@50527344[collection1] realtime
   [junit4]   2> 110982 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-24&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-24)} 0 13
   [junit4]   2> 110983 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[20 (25)]} 0 14
   [junit4]   2> 110984 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[20 (26)]} 0 14
   [junit4]   2> 110999 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 111000 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 111000 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 111001 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 111050 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@166f4e85[collection1] realtime
   [junit4]   2> 111051 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-23&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:21 (-23)} 0 81
   [junit4]   2> 111052 INFO  (WRITER5) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 111055 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (28)]} 0 78
   [junit4]   2> 111057 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (27)]} 0 87
   [junit4]   2> 111057 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (29)]} 0 74
   [junit4]   2> 111059 INFO  (WRITER5) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_4,generation=4}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_5,generation=5}
   [junit4]   2> 111060 INFO  (WRITER5) [    ] o.a.s.c.SolrCore newest commit generation = 5
   [junit4]   2> 111063 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 111086 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 111124 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 111125 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 111132 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4eaf12e2[collection1] main
   [junit4]   2> 111135 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4eaf12e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_8(5.4.0):c6/2:delGen=1) Uninverting(_c(5.4.0):C4/1) Uninverting(_e(5.4.0):C1) Uninverting(_f(5.4.0):C1) Uninverting(_d(5.4.0):C1)))}
   [junit4]   2> 111139 INFO  (WRITER5) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 111145 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 175
   [junit4]   2> 111166 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1644beec[collection1] realtime
   [junit4]   2> 111167 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-30&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:15 (-30)} 0 182
   [junit4]   2> 111168 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (31)]} 0 182
   [junit4]   2> 111170 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 111177 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@130ff04b[collection1] realtime
   [junit4]   2> 111178 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-32&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:21 (-32)} 0 126
   [junit4]   2> 111186 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7c8ce38d[collection1] realtime
   [junit4]   2> 111189 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-35&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-35)} 0 130
   [junit4]   2> 111189 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=33,id=4} DBQs=[DBQ{version=35,q=id:6}, DBQ{version=34,q=id:11}]
   [junit4]   2> 111190 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-34&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-34)} 0 133
   [junit4]   2> 111208 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@47fb8997[collection1] realtime
   [junit4]   2> 111209 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (36)]} 0 46
   [junit4]   2> 111217 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@67b12136[collection1] realtime
   [junit4]   2> 111219 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (33)]} 0 161
   [junit4]   2> 111222 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-38&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-38)} 0 52
   [junit4]   2> 111224 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-39&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-39)} 0 40
   [junit4]   2> 111226 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-40&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-40)} 0 35
   [junit4]   2> 111226 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 111235 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (43)]} 0 2
   [junit4]   2> 111234 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (42)]} 0 14
   [junit4]   2> 111234 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (45)]} 0 7
   [junit4]   2> 111233 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-37&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-37)} 0 64
   [junit4]   2> 111232 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (41)]} 0 22
   [junit4]   2> 111236 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (44)]} 0 3
   [junit4]   2> 111249 INFO  (WRITER3) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_5,generation=5}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_6,generation=6}
   [junit4]   2> 111249 INFO  (WRITER3) [    ] o.a.s.c.SolrCore newest commit generation = 6
   [junit4]   2> 111264 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@cab39ef[collection1] main
   [junit4]   2> 111265 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 111271 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@cab39ef[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_b(5.4.0):c4/2:delGen=1) Uninverting(_c(5.4.0):C4/2:delGen=1) Uninverting(_d(5.4.0):C1) Uninverting(_g(5.4.0):C1) Uninverting(_k(5.4.0):C3/1:delGen=1) Uninverting(_j(5.4.0):C2)))}
   [junit4]   2> 111272 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 81
   [junit4]   2> 111279 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3f9362f0[collection1] realtime
   [junit4]   2> 111281 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@35335489[collection1] realtime
   [junit4]   2> 111282 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-47&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-47)} 0 45
   [junit4]   2> 111284 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-48&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-48)} 0 45
   [junit4]   2> 111285 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@724bb435[collection1] realtime
   [junit4]   2> 111286 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-46&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-46)} 0 48
   [junit4]   2> 111286 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-50&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-50)} 0 47
   [junit4]   2> 111286 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-49&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-49)} 0 47
   [junit4]   2> 111287 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 111289 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (51)]} 0 49
   [junit4]   2> 111290 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (54)]} 0 4
   [junit4]   2> 111291 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (53)]} 0 6
   [junit4]   2> 111291 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (52)]} 0 5
   [junit4]   2> 111299 INFO  (WRITER3) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_6,generation=6}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_7,generation=7}
   [junit4]   2> 111300 INFO  (WRITER3) [    ] o.a.s.c.SolrCore newest commit generation = 7
   [junit4]   2> 111314 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@59ada21c[collection1] main
   [junit4]   2> 111316 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59ada21c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_l(5.4.0):c6) Uninverting(_j(5.4.0):C2/1:delGen=1) Uninverting(_m(5.4.0):C1) Uninverting(_p(5.4.0):C1) Uninverting(_n(5.4.0):C1) Uninverting(_o(5.4.0):C1)))}
   [junit4]   2> 111318 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 111318 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 41
   [junit4]   2> 111320 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-55&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-55)} 0 33
   [junit4]   2> 111322 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (56)]} 0 34
   [junit4]   2> 111322 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (57)]} 0 33
   [junit4]   2> 111329 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5293c8a6[collection1] realtime
   [junit4]   2> 111332 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (59)]} 0 40
   [junit4]   2> 111332 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (60)]} 0 40
   [junit4]   2> 111336 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-58&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-58)} 0 45
   [junit4]   2> 111344 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@79ef3e2b[collection1] realtime
   [junit4]   2> 111346 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (62)]} 0 24
   [junit4]   2> 111346 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (63)]} 0 24
   [junit4]   2> 111347 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (64)]} 0 24
   [junit4]   2> 111348 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (66)]} 0 15
   [junit4]   2> 111348 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (65)]} 0 14
   [junit4]   2> 111349 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (67)]} 0 9
   [junit4]   2> 111359 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6f4809f6[collection1] realtime
   [junit4]   2> 111360 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-69&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-69)} 0 12
   [junit4]   2> 111363 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7a6b136f[collection1] realtime
   [junit4]   2> 111365 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (72)]} 0 16
   [junit4]   2> 111366 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (73)]} 0 16
   [junit4]   2> 111382 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4b9681f7[collection1] realtime
   [junit4]   2> 111383 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-70&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-70)} 0 34
   [junit4]   2> 111387 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-61&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-61)} 0 67
   [junit4]   2> 111387 INFO  (WRITER5) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=68,id=21} DBQs=[DBQ{version=71,q=id:4}, DBQ{version=70,q=id:4}, DBQ{version=69,q=id:3}]
   [junit4]   2> 111398 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@15c5816b[collection1] realtime
   [junit4]   2> 111400 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-71&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-71)} 0 49
   [junit4]   2> 111401 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[21 (68)]} 0 48
   [junit4]   2> 111404 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-74&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-74)} 0 43
   [junit4]   2> 111405 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (75)]} 0 39
   [junit4]   2> 111405 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (76)]} 0 38
   [junit4]   2> 111406 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[16 (77)]} 0 23
   [junit4]   2> 111408 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (79)]} 0 6
   [junit4]   2> 111409 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (78)]} 0 17
   [junit4]   2> 111410 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (82)]} 0 3
   [junit4]   2> 111411 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (81)]} 0 3
   [junit4]   2> 111411 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (83)]} 0 5
   [junit4]   2> 111411 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (84)]} 0 3
   [junit4]   2> 111412 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[21 (80)]} 0 10
   [junit4]   2> 111412 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[17 (85)]} 0 2
   [junit4]   2> 111413 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (86)]} 0 3
   [junit4]   2> 111413 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (87)]} 0 2
   [junit4]   2> 111414 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (89)]} 0 2
   [junit4]   2> 111430 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@64c78d51[collection1] realtime
   [junit4]   2> 111432 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[9 (88)]} 0 19
   [junit4]   2> 111433 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[20 (92)]} 0 20
   [junit4]   2> 111434 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (93)]} 0 20
   [junit4]   2> 111448 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4154ca4[collection1] realtime
   [junit4]   2> 111449 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-91&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:20 (-91)} 0 35
   [junit4]   2> 111450 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-90&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-90)} 0 37
   [junit4]   2> 111451 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (94)]} 0 37
   [junit4]   2> 111452 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (95)]} 0 37
   [junit4]   2> 111479 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2c5737e0[collection1] realtime
   [junit4]   2> 111480 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-96&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-96)} 0 47
   [junit4]   2> 111481 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 111482 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (97)]} 0 48
   [junit4]   2> 111485 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (98)]} 0 50
   [junit4]   2> 111508 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (99)]} 0 58
   [junit4]   2> 111508 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (100)]} 0 56
   [junit4]   2> 111515 INFO  (WRITER2) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_7,generation=7}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_8,generation=8}
   [junit4]   2> 111515 INFO  (WRITER2) [    ] o.a.s.c.SolrCore newest commit generation = 8
   [junit4]   2> 111523 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1aabdbb3[collection1] main
   [junit4]   2> 111528 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1aabdbb3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_10(5.4.0):c15/3) Uninverting(_v(5.4.0):C1) Uninverting(_z(5.4.0):C2) Uninverting(_11(5.4.0):C1) Uninverting(_12(5.4.0):C1) Uninverting(_13(5.4.0):C1) Uninverting(_15(5.4.0):C1)))}
   [junit4]   2> 111535 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 111535 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 85
   [junit4]   2> 111539 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6acec49d[collection1] realtime
   [junit4]   2> 111539 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-101&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-101)} 0 87
   [junit4]   2> 111541 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (102)]} 0 58
   [junit4]   2> 111550 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@18d4b89e[collection1] realtime
   [junit4]   2> 111553 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-104&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:16 (-104)} 0 67
   [junit4]   2> 111554 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-103&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:16 (-103)} 0 71
   [junit4]   2> 111554 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[16 (105)]} 0 45
   [junit4]   2> 111554 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[16 (106)]} 0 45
   [junit4]   2> 111566 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@18e16126[collection1] realtime
   [junit4]   2> 111568 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (109)]} 0 27
   [junit4]   2> 111569 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (110)]} 0 15
   [junit4]   2> 111570 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (108)]} 0 29
   [junit4]   2> 111575 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-107&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:16 (-107)} 0 36
   [junit4]   2> 111581 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@727dc077[collection1] realtime
   [junit4]   2> 111584 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-113&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-113)} 0 27
   [junit4]   2> 111586 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-112&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:16 (-112)} 0 30
   [junit4]   2> 111588 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[19 (114)]} 0 19
   [junit4]   2> 111613 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1b2256ec[collection1] realtime
   [junit4]   2> 111613 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-115&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-115)} 0 43
   [junit4]   2> 111616 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-116&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-116)} 0 44
   [junit4]   2> 111619 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@38a1c177[collection1] realtime
   [junit4]   2> 111622 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-118&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-118)} 0 37
   [junit4]   2> 111622 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-111&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-111)} 0 68
   [junit4]   2> 111624 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (120)]} 0 35
   [junit4]   2> 111624 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (119)]} 0 35
   [junit4]   2> 111625 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[16 (122)]} 0 9
   [junit4]   2> 111625 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-117&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:17 (-117)} 0 50
   [junit4]   2> 111644 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 111646 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 111653 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 111657 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 111664 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2f07f387[collection1] realtime
   [junit4]   2> 111665 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-124&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-124)} 0 42
   [junit4]   2> 111674 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[21 (125)]} 0 50
   [junit4]   2> 111675 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[21 (126)]} 0 50
   [junit4]   2> 111676 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (127)]} 0 50
   [junit4]   2> 111685 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@442fec83[collection1] realtime
   [junit4]   2> 111701 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (128)]} 0 74
   [junit4]   2> 111708 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3f8c1384[collection1] realtime
   [junit4]   2> 111709 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-123&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-123)} 0 82
   [junit4]   2> 111710 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-121&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-121)} 0 82
   [junit4]   2> 111711 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (129)]} 0 45
   [junit4]   2> 111719 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 111719 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 111720 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001/]
   [junit4]   2> 111720 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e3553f9
   [junit4]   2> 111732 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001/index/
   [junit4]   2> 111739 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7aabd543[collection1] realtime
   [junit4]   2> 111740 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-130&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-130)} 0 65
   [junit4]   2> 111742 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (131)]} 0 64
   [junit4]   2> 111752 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7b87d6c5[collection1] realtime
   [junit4]   2> 111760 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (133)]} 0 58
   [junit4]   2> 111764 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 111767 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 111767 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 111767 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 111768 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 111768 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 111768 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 111769 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 111769 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 111769 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_E5F76943B73D63E3-001/init-core-data-001/
   [junit4]   2> 111775 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111788 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@182a483f[collection1] realtime
   [junit4]   2> 111789 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-132&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:18 (-132)} 0 111
   [junit4]   2> 111789 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-134&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-134)} 0 79
   [junit4]   2> 111791 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111793 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4f85ac01[collection1] realtime
   [junit4]   2> 111793 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111795 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111803 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-136&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-136)} 0 92
   [junit4]   2> 111803 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=135,id=8} DBQs=[DBQ{version=136,q=id:8}]
   [junit4]   2> 111824 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/admin/threads,/schema,/get,/admin/system,/admin/luke,/admin/file,/update/json/docs,/update/csv,/admin/properties,/admin/mbeans,standard,/admin/logging,/replication,/update/json,/admin/segments,/admin/plugins,/admin/ping,/config
   [junit4]   2> 111826 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 111831 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 111835 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 111936 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@794dca09[collection1] realtime
   [junit4]   2> 111937 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (137)]} 0 196
   [junit4]   2> 111938 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[20 (139)]} 0 173
   [junit4]   2> 111938 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (138)]} 0 195
   [junit4]   2> 111945 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2c2569a1[collection1] realtime
   [junit4]   2> 111946 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (135)]} 0 231
   [junit4]   2> 111946 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@fd6b1d6[collection1] main
   [junit4]   2> 111946 WARN  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 111952 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 111952 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 111953 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 111953 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (142)]} 0 149
   [junit4]   2> 111949 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (141)]} 0 159
   [junit4]   2> 111949 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (143)]} 0 10
   [junit4]   2> 111948 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[17 (144)]} 0 9
   [junit4]   2> 111953 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 111952 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-140&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-140)} 0 163
   [junit4]   2> 111956 INFO  (searcherExecutor-203-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@fd6b1d6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1i(5.4.0):c11/3) Uninverting(_v(5.4.0):C1) Uninverting(_1a(5.4.0):C1) Uninverting(_1l(5.4.0):C1) Uninverting(_1m(5.4.0):C1)))}
   [junit4]   2> 111956 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 112016 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2dec8e27[collection1] realtime
   [junit4]   2> 112018 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (146)]} 0 72
   [junit4]   2> 112019 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-145&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:17 (-145)} 0 80
   [junit4]   2> 112029 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@29859ec3[collection1] realtime
   [junit4]   2> 112031 INFO  (WRITER6) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 112032 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[9 (148)]} 0 78
   [junit4]   2> 112035 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-147&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-147)} 0 82
   [junit4]   2> 112038 INFO  (WRITER6) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_8,generation=8}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5f5c9a08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3884b421),segFN=segments_9,generation=9}
   [junit4]   2> 112039 INFO  (WRITER6) [    ] o.a.s.c.SolrCore newest commit generation = 9
   [junit4]   2> 112040 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@d9e5aba[collection1] main
   [junit4]   2> 112042 INFO  (searcherExecutor-198-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@d9e5aba[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1i(5.4.0):c11/3:delGen=1) Uninverting(_v(5.4.0):C1) Uninverting(_1a(5.4.0):C1) Uninverting(_1l(5.4.0):C1) Uninverting(_1m(5.4.0):C1) Uninverting(_1n(5.4.0):C2/1:delGen=1) Uninverting(_1q(5.4.0):C1) Uninverting(_1r(5.4.0):C1)))}
   [junit4]   2> 112044 INFO  (WRITER6) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 112047 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@33bcfb09[collection1] realtime
   [junit4]   2> 112049 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[16 (150)]} 0 93
   [junit4]   2> 112049 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-149&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:18 (-149)} 0 94
   [junit4]   2> 112050 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 95
   [junit4]   2> 112058 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 112071 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.VersionInfo Found MAX value 139 from Terms for _version_ in index
   [junit4]   2> 112074 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.UpdateLog Took 25.0ms to seed version buckets with highest version 150
   [junit4]   2> 112079 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (151)]} 0 60
   [junit4]   2> 112080 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (153)]} 0 47
   [junit4]   2> 112081 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[18 (152)]} 0 61
   [junit4]   2> 112082 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63E3]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Creating new IndexWriter...
   [junit4]   2> 112082 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[E5F76943B73D63

[...truncated too long message...]

ter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 5263822 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[E5F76943B73D63E3]-worker) [n:127.0.0.1:40450_ c:addReplicaColl s:shard1 r:core_node2 x:addReplicaColl_shard1_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 18, 2015 12:45:13 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=Direct), text=PostingsFormat(name=LuceneFixedGap), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=de_AT, timezone=Africa/Dakar
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=116604016,total=534773760
   [junit4]   2> NOTE: All tests run in this JVM: [TestDistributedMissingSort, ConnectionManagerTest, TestAuthorizationFramework, SaslZkACLProviderTest, TestMaxScoreQueryParser, TestSearchPerf, DirectSolrConnectionTest, TestReloadDeadlock, TestNoOpRegenerator, SpellCheckComponentTest, DisMaxRequestHandlerTest, HdfsThreadLeakTest, TestElisionMultitermQuery, TestHashPartitioner, BasicAuthIntegrationTest, TestManagedSynonymFilterFactory, AlternateDirectoryTest, TestImplicitCoreProperties, CursorPagingTest, TestClassNameShortening, TestSuggestSpellingConverter, SolrCmdDistributorTest, NoCacheHeaderTest, AssignTest, TestSearcherReuse, TestCursorMarkWithoutUniqueKey, TestFreeTextSuggestions, JsonLoaderTest, TestFieldCollectionResource, TestHighFrequencyDictionaryFactory, TestLMDirichletSimilarityFactory, DistanceFunctionTest, TestManagedSchemaDynamicFieldResource, RAMDirectoryFactoryTest, TestCustomSort, PreAnalyzedFieldTest, TestManagedResource, TestFastLRUCache, ZkStateReaderTest, TestCollapseQParserPlugin, TestJsonRequest, TestJsonFacets, DistributedTermsComponentTest, TestMacros, CloudExitableDirectoryReaderTest, FieldAnalysisRequestHandlerTest, TestCoreDiscovery, ExitableDirectoryReaderTest, DistributedMLTComponentTest, OverseerRolesTest, TestSolrConfigHandler, AtomicUpdatesTest, SSLMigrationTest, BadComponentTest, TestTolerantSearch, TestDistributedGrouping, DistanceUnitsTest, TestDefaultStatsCache, TestWriterPerf, TestIndexSearcher, TestLRUStatsCache, RuleEngineTest, TestConfigSetProperties, JSONWriterTest, CopyFieldTest, SpatialFilterTest, FileUtilsTest, CleanupOldIndexTest, CacheHeaderTest, TestNonDefinedSimilarityFactory, ChaosMonkeySafeLeaderTest, TestManagedResourceStorage, DistributedFacetPivotSmallTest, SliceStateTest, TestConfigOverlay, DataDrivenBlockJoinTest, TestExpandComponent, OutOfBoxZkACLAndCredentialsProvidersTest, RequiredFieldsTest, ReturnFieldsTest, ZkSolrClientTest, TestConfigReload, TestFileDictionaryLookup, AnalyticsQueryTest, IndexSchemaRuntimeFieldTest, UpdateParamsTest, TestMissingGroups, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, ClusterStateUpdateTest, LeaderElectionTest, TestRandomDVFaceting, AutoCommitTest, TestRecovery, ZkControllerTest, TestRealTimeGet, TestReload, TestStressVersions, HardAutoCommitTest, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, StatsComponentTest, QueryElevationComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, TestFiltering, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, TestBadConfig, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTSTTest, SuggesterTest, TestStressLucene, SuggesterWFSTTest, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DocValuesMultiTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, LukeRequestHandlerTest, StandardRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, DocumentBuilderTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, MoreLikeThisHandlerTest, TestArbitraryIndexDir, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, TestCollationField, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestPHPSerializedResponseWriter, PingRequestHandlerTest, TestComponentsName, TestLFUCache, ScriptEngineTest, URLClassifyProcessorTest, TestSystemIdResolver, DOMUtilTest, TestLRUCache, ZkNodePropsTest, UUIDFieldTest, SolrTestCaseJ4Test, TestEmbeddedSolrServerConstructors, ActionThrottleTest, AsyncCallRequestStatusResponseTest, CollectionReloadTest, DeleteLastCustomShardedReplicaTest, DistribDocExpirationUpdateProcessorTest, DistributedQueueTest, HttpPartitionTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCollectionAPI, TestRandomRequestDistribution, TestRebalanceLeaders, TestRequestStatusCollectionAPI, HdfsBasicDistributedZk2Test, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [533/533] on J1 in 2006.57s, 1 test, 1 error <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 533 suites, 2114 tests, 2 errors, 1 failure, 26 ignored (14 assumptions)

Total time: 279 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1064833316 bytes
Compression is 0.0%
Took 3 min 10 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 930 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/930/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=101104, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=101104, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40599: collection already exists: awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([456BEE68EFC74744]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1439836692008,generation=2,filelist=[_1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_LuceneVarGapFixedInterval_0.doc, _5_LuceneVarGapFixedInterval_0.tib, _5_LuceneVarGapFixedInterval_0.tiv, _6.cfe, _6.cfs, _6.si, segments_2]}]> but was:<[{indexVersion=1439836692008,generation=2,filelist=[_1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_LuceneVarGapFixedInterval_0.doc, _5_LuceneVarGapFixedInterval_0.tib, _5_LuceneVarGapFixedInterval_0.tiv, _6.cfe, _6.cfs, _6.si, segments_2]}, {indexVersion=1439836692008,generation=3,filelist=[_5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_LuceneVarGapFixedInterval_0.doc, _5_LuceneVarGapFixedInterval_0.tib, _5_LuceneVarGapFixedInterval_0.tiv, _6.cfe, _6.cfs, _6.si, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_LuceneVarGapFixedInterval_0.doc, _7_LuceneVarGapFixedInterval_0.tib, _7_LuceneVarGapFixedInterval_0.tiv, _8.cfe, _8.cfs, _8.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1439836692008,generation=2,filelist=[_1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_LuceneVarGapFixedInterval_0.doc, _5_LuceneVarGapFixedInterval_0.tib, _5_LuceneVarGapFixedInterval_0.tiv, _6.cfe, _6.cfs, _6.si, segments_2]}]> but was:<[{indexVersion=1439836692008,generation=2,filelist=[_1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_LuceneVarGapFixedInterval_0.doc, _5_LuceneVarGapFixedInterval_0.tib, _5_LuceneVarGapFixedInterval_0.tiv, _6.cfe, _6.cfs, _6.si, segments_2]}, {indexVersion=1439836692008,generation=3,filelist=[_5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_LuceneVarGapFixedInterval_0.doc, _5_LuceneVarGapFixedInterval_0.tib, _5_LuceneVarGapFixedInterval_0.tiv, _6.cfe, _6.cfs, _6.si, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_LuceneVarGapFixedInterval_0.doc, _7_LuceneVarGapFixedInterval_0.tib, _7_LuceneVarGapFixedInterval_0.tiv, _8.cfe, _8.cfs, _8.si, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([456BEE68EFC74744:60BCF5589F8F4947]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=36948, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=36948, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56766/frj: Could not find collection : awholynewstresscollection_collection5_1
	at __randomizedtesting.SeedInfo.seed([456BEE68EFC74744]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=5279, name=WRITER8, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5279, name=WRITER8, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
	at __randomizedtesting.SeedInfo.seed([456BEE68EFC74744:78EA4AC9BB8CA406]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([456BEE68EFC74744]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1673)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1530)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:779)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1088)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:705)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
	... 21 more




Build Log:
[...truncated 10253 lines...]
   [junit4] Suite: org.apache.solr.search.TestReloadDeadlock
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001
   [junit4]   2> 426673 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 426673 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 426673 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 426674 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 426674 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 426674 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 426706 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 426721 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 426751 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 426752 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 426758 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 427006 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 427007 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 427009 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 427012 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 427023 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 427023 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 427023 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 427040 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 2125804009
   [junit4]   2> 427040 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 427040 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
   [junit4]   2> 427040 WARN  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib).
   [junit4]   2> 427050 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 427053 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 427053 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 427054 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 427054 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 427054 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 427056 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=shard1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, name=collection1, collection=collection1, configSetProperties=configsetprops.json, transient=false, config=solrconfig-tlog.xml, schema=schema15.xml, instanceDir=collection1, dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001, loadOnStartup=true}
   [junit4]   2> 427064 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 427068 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 427068 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 427068 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 427095 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 427111 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 427158 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 427158 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 427166 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 427429 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 427431 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 427432 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 427434 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 427444 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 427444 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 427444 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 427445 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 427445 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b59543d
   [junit4]   2> 427445 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001
   [junit4]   2> 427445 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001/index/
   [junit4]   2> 427445 WARN  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 427446 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001/index
   [junit4]   2> 427446 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 427447 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_1,generation=1}
   [junit4]   2> 427447 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 427451 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001/
   [junit4]   2> 427451 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 427452 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 427452 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 427452 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 427452 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 427452 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 427453 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 427453 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 427453 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 427456 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427458 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427459 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427460 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427465 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/segments,/admin/properties,/schema,/admin/logging,/update/csv,/admin/mbeans,/update,standard,/update/json,/replication,/admin/plugins,/config,/admin/system,/admin/file,/admin/ping,/get,/update/json/docs,/admin/luke
   [junit4]   2> 427466 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 427466 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 427466 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 427467 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 427467 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 427468 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=5, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 427468 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_1,generation=1}
   [junit4]   2> 427468 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 427468 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5de22625[collection1] main
   [junit4]   2> 427469 WARN  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 427469 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 427469 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 427469 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 427469 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 427470 INFO  (searcherExecutor-1029-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5de22625[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 427470 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 427470 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 427470 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 427470 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1509776355497082880
   [junit4]   2> 427472 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1509776355497082880
   [junit4]   2> 427473 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 427474 INFO  (SUITE-TestReloadDeadlock-seed#[456BEE68EFC74744]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 427478 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testReloadDeadlock
   [junit4]   2> 427479 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 427480 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7223cf2[collection1] realtime
   [junit4]   2> 427480 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=*:* (-9223372036854775807)} 0 1
   [junit4]   2> 427481 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 427481 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_2,generation=2}
   [junit4]   2> 427481 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 427482 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@f12e69a[collection1] main
   [junit4]   2> 427482 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 427483 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@f12e69a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 427483 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 3
   [junit4]   2> 427488 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 427490 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 427490 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 427490 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 427491 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 2
   [junit4]   2> 427495 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-1&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-1)} 0 4
   [junit4]   2> 427495 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-2&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-2)} 0 4
   [junit4]   2> 427496 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-3&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-3)} 0 4
   [junit4]   2> 427499 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (4)]} 0 7
   [junit4]   2> 427505 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1b394bed[collection1] realtime
   [junit4]   2> 427505 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-5&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-5)} 0 12
   [junit4]   2> 427506 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-7&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-7)} 0 10
   [junit4]   2> 427507 INFO  (WRITER9) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=6,id=8} DBQs=[DBQ{version=7,q=id:11}]
   [junit4]   2> 427508 INFO  (WRITER4) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 427568 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@455b9690[collection1] realtime
   [junit4]   2> 427569 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (6)]} 0 73
   [junit4]   2> 427569 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (8)]} 0 72
   [junit4]   2> 427569 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (12)]} 0 72
   [junit4]   2> 427572 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (10)]} 0 74
   [junit4]   2> 427572 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (11)]} 0 75
   [junit4]   2> 427573 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (9)]} 0 76
   [junit4]   2> 427573 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (14)]} 0 73
   [junit4]   2> 427576 INFO  (WRITER4) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_3,generation=3}
   [junit4]   2> 427576 INFO  (WRITER4) [    ] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 427583 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 427584 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 427584 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 427584 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 427612 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 427613 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@54dccd3f[collection1] main
   [junit4]   2> 427613 INFO  (WRITER4) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 427613 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@54dccd3f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(5.4.0):c1) Uninverting(_2(5.4.0):c1) Uninverting(_5(5.4.0):c1) Uninverting(_4(5.4.0):c1) Uninverting(_6(5.4.0):c1) Uninverting(_7(5.4.0):c1)))}
   [junit4]   2> 427614 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 114
   [junit4]   2> 427624 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@25987211[collection1] realtime
   [junit4]   2> 427624 INFO  (WRITER6) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=13,id=5} DBQs=[DBQ{version=15,q=id:10}]
   [junit4]   2> 427624 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-15&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-15)} 0 124
   [junit4]   2> 427635 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 427637 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7915d36a[collection1] realtime
   [junit4]   2> 427638 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (16)]} 0 132
   [junit4]   2> 427639 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (13)]} 0 137
   [junit4]   2> 427640 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (17)]} 0 134
   [junit4]   2> 427641 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (28)]} 0 0
   [junit4]   2> 427641 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (18)]} 0 71
   [junit4]   2> 427640 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[9 (22)]} 0 67
   [junit4]   2> 427639 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (26)]} 0 14
   [junit4]   2> 427639 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (21)]} 0 67
   [junit4]   2> 427642 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (20)]} 0 72
   [junit4]   2> 427643 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (23)]} 0 70
   [junit4]   2> 427644 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (25)]} 0 24
   [junit4]   2> 427644 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (24)]} 0 70
   [junit4]   2> 427644 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (19)]} 0 74
   [junit4]   2> 427644 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (27)]} 0 5
   [junit4]   2> 427665 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 427665 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 427671 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 427680 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6a07daa4[collection1] realtime
   [junit4]   2> 427680 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-29&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:14 (-29)} 0 39
   [junit4]   2> 427681 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 427683 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (33)]} 0 40
   [junit4]   2> 427684 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (32)]} 0 42
   [junit4]   2> 427685 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (31)]} 0 43
   [junit4]   2> 427685 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (36)]} 0 40
   [junit4]   2> 427686 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (34)]} 0 43
   [junit4]   2> 427686 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (39)]} 0 41
   [junit4]   2> 427688 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (30)]} 0 47
   [junit4]   2> 427696 INFO  (WRITER3) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_3,generation=3}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_4,generation=4}
   [junit4]   2> 427696 INFO  (WRITER3) [    ] o.a.s.c.SolrCore newest commit generation = 4
   [junit4]   2> 427756 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@16b61dca[collection1] main
   [junit4]   2> 427757 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 427758 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@16b61dca[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_q(5.4.0):c11/3) Uninverting(_n(5.4.0):c1) Uninverting(_p(5.4.0):c2) Uninverting(_o(5.4.0):c1)))}
   [junit4]   2> 427758 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 115
   [junit4]   2> 427759 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-38&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:14 (-38)} 0 115
   [junit4]   2> 427761 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-35&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-35)} 0 116
   [junit4]   2> 427771 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5f51f251[collection1] realtime
   [junit4]   2> 427771 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-37&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-37)} 0 126
   [junit4]   2> 427772 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (40)]} 0 91
   [junit4]   2> 427825 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@16940093[collection1] realtime
   [junit4]   2> 427826 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-41&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-41)} 0 142
   [junit4]   2> 427827 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (42)]} 0 142
   [junit4]   2> 427830 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@179a97e5[collection1] realtime
   [junit4]   2> 427830 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-44&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-44)} 0 145
   [junit4]   2> 427831 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-43&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-43)} 0 145
   [junit4]   2> 427832 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (45)]} 0 145
   [junit4]   2> 427835 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@257688d[collection1] realtime
   [junit4]   2> 427837 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (48)]} 0 76
   [junit4]   2> 427837 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (47)]} 0 148
   [junit4]   2> 427837 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (49)]} 0 68
   [junit4]   2> 427838 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-46&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-46)} 0 149
   [junit4]   2> 427890 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 427891 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 427893 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 427895 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 427898 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@670b0372[collection1] realtime
   [junit4]   2> 427899 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-50&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-50)} 0 127
   [junit4]   2> 427899 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (51)]} 0 127
   [junit4]   2> 427900 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (52)]} 0 126
   [junit4]   2> 427905 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1be14805[collection1] realtime
   [junit4]   2> 427906 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (54)]} 0 78
   [junit4]   2> 427906 INFO  (WRITER4) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 427907 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-53&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-53)} 0 80
   [junit4]   2> 427907 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (55)]} 0 75
   [junit4]   2> 427910 INFO  (WRITER4) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_4,generation=4}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_5,generation=5}
   [junit4]   2> 427910 INFO  (WRITER4) [    ] o.a.s.c.SolrCore newest commit generation = 5
   [junit4]   2> 427910 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 427910 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 427910 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1d613de7[collection1] main
   [junit4]   2> 427911 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001/]
   [junit4]   2> 427911 INFO  (WRITER4) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 427911 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b59543d
   [junit4]   2> 427911 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1d613de7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_w(5.4.0):c10/1:delGen=1) Uninverting(_x(5.4.0):c1) Uninverting(_y(5.4.0):c1) Uninverting(_z(5.4.0):c2)))}
   [junit4]   2> 427912 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001/index/
   [junit4]   2> 427912 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 81
   [junit4]   2> 427914 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7b0df949[collection1] realtime
   [junit4]   2> 427914 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-56&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-56)} 0 82
   [junit4]   2> 427915 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_456BEE68EFC74744-001/init-core-data-001/
   [junit4]   2> 427915 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (57)]} 0 78
   [junit4]   2> 427915 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (58)]} 0 78
   [junit4]   2> 427916 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (59)]} 0 78
   [junit4]   2> 427917 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 427918 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 427918 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 427918 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 427918 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 427918 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 427919 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 427919 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 427920 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 427926 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4d373a3e[collection1] realtime
   [junit4]   2> 427926 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-60&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:14 (-60)} 0 87
   [junit4]   2> 427927 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@30fc1774[collection1] realtime
   [junit4]   2> 427927 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427927 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-61&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-61)} 0 27
   [junit4]   2> 427928 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (62)]} 0 28
   [junit4]   2> 427928 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (63)]} 0 28
   [junit4]   2> 427928 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (65)]} 0 21
   [junit4]   2> 427928 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (66)]} 0 20
   [junit4]   2> 427928 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (68)]} 0 14
   [junit4]   2> 427928 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (64)]} 0 21
   [junit4]   2> 427929 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (70)]} 0 13
   [junit4]   2> 427929 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (67)]} 0 16
   [junit4]   2> 427931 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427932 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427934 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@670bfbfa[collection1] realtime
   [junit4]   2> 427934 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 427934 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-69&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:14 (-69)} 0 18
   [junit4]   2> 427934 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (71)]} 0 18
   [junit4]   2> 427935 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (72)]} 0 8
   [junit4]   2> 427942 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5a4246be[collection1] realtime
   [junit4]   2> 427943 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-73&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-73)} 0 15
   [junit4]   2> 427943 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (74)]} 0 15
   [junit4]   2> 427944 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/segments,/admin/properties,/schema,/admin/logging,/update/csv,/admin/mbeans,/update,standard,/update/json,/replication,/admin/plugins,/config,/admin/system,/admin/file,/admin/ping,/get,/update/json/docs,/admin/luke
   [junit4]   2> 427945 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 427946 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 427946 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 428054 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@65467412[collection1] main
   [junit4]   2> 428054 WARN  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 428055 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 428055 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 428055 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 428056 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 428056 INFO  (searcherExecutor-1034-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@65467412[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_12(5.4.0):c11/7) Uninverting(_14(5.4.0):c6/1) Uninverting(_16(5.4.0):c1) Uninverting(_15(5.4.0):c1) Uninverting(_17(5.4.0):c1)))}
   [junit4]   2> 428057 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 428059 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@17894e52[collection1] realtime
   [junit4]   2> 428059 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-75&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-75)} 0 131
   [junit4]   2> 428060 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (76)]} 0 131
   [junit4]   2> 428060 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (77)]} 0 131
   [junit4]   2> 428061 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (79)]} 0 131
   [junit4]   2> 428064 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (80)]} 0 134
   [junit4]   2> 428072 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3485ad67[collection1] realtime
   [junit4]   2> 428073 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-78&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-78)} 0 143
   [junit4]   2> 428074 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4f90e7a4[collection1] realtime
   [junit4]   2> 428075 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-82&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:14 (-82)} 0 140
   [junit4]   2> 428077 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@197cd39e[collection1] realtime
   [junit4]   2> 428077 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-84&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-84)} 0 142
   [junit4]   2> 428077 INFO  (WRITER10) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=83,id=1} DBQs=[DBQ{version=84,q=id:0}]
   [junit4]   2> 428100 INFO  (WRITER10) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6f7e807e[collection1] realtime
   [junit4]   2> 428100 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-81&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-81)} 0 171
   [junit4]   2> 428101 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (86)]} 0 157
   [junit4]   2> 428101 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (83)]} 0 166
   [junit4]   2> 428102 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 428102 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (85)]} 0 158
   [junit4]   2> 428109 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.VersionInfo Found MAX value 74 from Terms for _version_ in index
   [junit4]   2> 428111 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.UpdateLog Took 8.0ms to seed version buckets with highest version 86
   [junit4]   2> 428111 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Creating new IndexWriter...
   [junit4]   2> 428111 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Waiting until IndexWriter is unused... core=collection1
   [junit4]   2> 428111 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Closing old IndexWriter... core=collection1
   [junit4]   2> 428165 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_5,generation=5}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_6,generation=6}
   [junit4]   2> 428165 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 6
   [junit4]   2> 428166 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=88.4013671875, floorSegmentMB=0.3203125, forceMergeDeletesPctAllowed=21.942637308492678, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2452028314340395
   [junit4]   2> 428169 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_6,generation=6}
   [junit4]   2> 428169 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 6
   [junit4]   2> 428170 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 428171 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (87)]} 0 111
   [junit4]   2> 428180 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@602b1d79[collection1] main
   [junit4]   2> 428181 INFO  (searcherExecutor-1034-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@602b1d79[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1e(5.4.0):c10/2:delGen=1) Uninverting(_1c(5.4.0):c1) Uninverting(_1f(5.4.0):c1) Uninverting(_1g(5.4.0):c1)))}
   [junit4]   2> 428181 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.CoreContainer replacing core: collection1
   [junit4]   2> 428182 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@77796fa7
   [junit4]   2> 428184 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@55d462b2[collection1] realtime
   [junit4]   2> 428188 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-88&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:2 (-88)} 0 127
   [junit4]   2> 428188 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (91)]} 0 123
   [junit4]   2> 428189 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (90)]} 0 128
   [junit4]   2> 428190 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (89)]} 0 129
   [junit4]   2> 428211 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@373d019e[collection1] realtime
   [junit4]   2> 428216 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=22,adds=22,deletesById=0,deletesByQuery=11,errors=1,cumulative_adds=60,cumulative_deletesById=0,cumulative_deletesByQuery=28,cumulative_errors=1,transaction_logs_total_size=1536,transaction_logs_total_number=3}
   [junit4]   2> 428216 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 428222 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-92&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-92)} 0 148
   [junit4]   2> 428224 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (93)]} 0 149
   [junit4]   2> 428225 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (96)]} 0 123
   [junit4]   2> 428225 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (95)]} 0 124
   [junit4]   2> 428225 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (94)]} 0 147
   [junit4]   2> 428226 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (97)]} 0 122
   [junit4]   2> 428235 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2bc18cd5[collection1] realtime
   [junit4]   2> 428236 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-98&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:2 (-98)} 0 131
   [junit4]   2> 428236 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (99)]} 0 65
   [junit4]   2> 428236 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (100)]} 0 48
   [junit4]   2> 428293 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2f9075f6[collection1] realtime
   [junit4]   2> 428293 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 428293 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-101&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-101)} 0 104
   [junit4]   2> 428294 INFO  (WRITER2) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_6,generation=6}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_7,generation=7}
   [junit4]   2> 428295 INFO  (WRITER2) [    ] o.a.s.c.SolrCore newest commit generation = 7
   [junit4]   2> 428295 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3c762578[collection1] main
   [junit4]   2> 428295 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 428296 INFO  (searcherExecutor-1034-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3c762578[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1e(5.4.0):c10/5:delGen=2) Uninverting(_1h(5.4.0):c1) Uninverting(_1j(5.4.0):c1) Uninverting(_1l(5.4.0):c2/1:delGen=1) Uninverting(_1n(5.4.0):c1) Uninverting(_1o(5.4.0):c1)))}
   [junit4]   2> 428297 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 107
   [junit4]   2> 428301 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-102&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-102)} 0 110
   [junit4]   2> 428302 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-103&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-103)} 0 78
   [junit4]   2> 428304 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1b23dbc3[collection1] realtime
   [junit4]   2> 428305 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-104&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-104)} 0 79
   [junit4]   2> 428306 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (107)]} 0 80
   [junit4]   2> 428306 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (111)]} 0 69
   [junit4]   2> 428307 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (108)]} 0 80
   [junit4]   2> 428307 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (105)]} 0 82
   [junit4]   2> 428307 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (110)]} 0 70
   [junit4]   2> 428307 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[9 (109)]} 0 70
   [junit4]   2> 428307 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (106)]} 0 82
   [junit4]   2> 428326 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3a9298f4[collection1] realtime
   [junit4]   2> 428327 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (114)]} 0 25
   [junit4]   2> 428331 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3549f0ba[collection1] realtime
   [junit4]   2> 428332 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (115)]} 0 27
   [junit4]   2> 428336 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@40c425dc[collection1] realtime
   [junit4]   2> 428336 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-116&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-116)} 0 31
   [junit4]   2> 428337 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-117&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-117)} 0 31
   [junit4]   2> 428338 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-112&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:15 (-112)} 0 43
   [junit4]   2> 428337 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-113&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-113)} 0 34
   [junit4]   2> 428338 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (118)]} 0 31
   [junit4]   2> 428338 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (119)]} 0 31
   [junit4]   2> 428342 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@721fbd9f[collection1] realtime
   [junit4]   2> 428342 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-120&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-120)} 0 34
   [junit4]   2> 428343 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (121)]} 0 35
   [junit4]   2> 428343 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (124)]} 0 10
   [junit4]   2> 428343 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (126)]} 0 5
   [junit4]   2> 428342 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 428343 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (125)]} 0 6
   [junit4]   2> 428348 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (122)]} 0 40
   [junit4]   2> 428349 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (123)]} 0 22
   [junit4]   2> 428356 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_7,generation=7}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4846b836 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ae8064c),segFN=segments_8,generation=8}
   [junit4]   2> 428356 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 8
   [junit4]   2> 428367 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 428368 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 428368 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 428368 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 428409 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[456BEE68EFC74744]) [    x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 428413 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@66ba3637[collection1] main
   [junit4]   2> 428414 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 428414 INFO  (searcherExecutor-1034-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@66ba3637[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1e(5.4.0):c10/6:delGen=3) Uninverting(_1h(5.4.0):c1) Uninverting(_1n(5.4.0):c1) Uninverting(_1r(5.4.0):c1) Uninverting(_1s(5.4.0):c2/1:delGen=1) Uninverting(_1w(5.4.0):c4/2:delGen=1) Uninverting(_1x(5.4.0):c1) Uninverting(_1y(5.4.0):c1)))}
   [junit4]   2> 428415 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 107
   [junit4]   2> 428417 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@37356f4e[collection1] realtime
   [junit4]   2> 428417 INFO  (WRITER9

[...truncated too long message...]

eMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3486400 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[456BEE68EFC74744]-worker) [n:127.0.0.1:60081_frj c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 17, 2015 2:59:53 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {intDefault=PostingsFormat(name=Memory doPackFST= true), range_facet_l_dv=PostingsFormat(name=Direct), multiDefault=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), text=Lucene50(blocksize=128), other_tl1=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true), id=PostingsFormat(name=Direct), a_i1=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=Memory doPackFST= true)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ar_EG, timezone=America/Juneau
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=266614040,total=525860864
   [junit4]   2> NOTE: All tests run in this JVM: [DeleteReplicaTest, TestFastOutputStream, MoreLikeThisHandlerTest, TestGroupingSearch, TestReload, HdfsSyncSliceTest, OverseerCollectionProcessorTest, TestSolrCloudWithKerberosAlt, SampleTest, SpatialHeatmapFacetsTest, TestQueryTypes, HardAutoCommitTest, BinaryUpdateRequestHandlerTest, PolyFieldTest, DocExpirationUpdateProcessorFactoryTest, TestQueryUtils, SpatialRPTFieldTypeTest, TestReloadAndDeleteDocs, SynonymTokenizerTest, MultiThreadedOCPTest, TestSchemaManager, TestSmileRequest, ZkStateWriterTest, ChangedSchemaMergeTest, TestExtendedDismaxParser, CircularListTest, TestSurroundQueryParser, StressHdfsTest, TestBlendedInfixSuggestions, LoggingHandlerTest, TestDynamicFieldResource, BadIndexSchemaTest, TestStressReorder, HdfsLockFactoryTest, CoreAdminRequestStatusTest, TestBM25SimilarityFactory, CustomCollectionTest, SOLR749Test, TestJettySolrRunner, TestFieldTypeCollectionResource, TestXIncludeConfig, OverseerTest, TestPerFieldSimilarity, DOMUtilTest, NotRequiredUniqueKeyTest, StatsComponentTest, DirectSolrSpellCheckerTest, AnalysisErrorHandlingTest, TestRequestStatusCollectionAPI, AsyncCallRequestStatusResponseTest, TestDistribIDF, TestRestoreCore, ConcurrentDeleteAndCreateCollectionTest, OutputWriterTest, TestShardHandlerFactory, FileBasedSpellCheckerTest, CoreMergeIndexesAdminHandlerTest, RankQueryTest, TestSimpleTrackingShardHandler, SmileWriterTest, CollectionReloadTest, TestCollationField, TestDefaultSimilarityFactory, TestDocBasedVersionConstraints, DistribCursorPagingTest, TestCSVResponseWriter, DistributedQueryComponentCustomSortTest, HdfsUnloadDistributedZkTest, QueryParsingTest, RegexBoostProcessorTest, SolrIndexSplitterTest, TestDocSet, TestHdfsUpdateLog, TestCloudInspectUtil, LeaderElectionTest, TestFieldTypeResource, TestCryptoKeys, BasicDistributedZk2Test, TestSolrQueryParserResource, XmlUpdateRequestHandlerTest, SimpleCollectionCreateDeleteTest, TestSchemaSimilarityResource, SolrInfoMBeanTest, PreAnalyzedUpdateProcessorTest, SyncSliceTest, OpenCloseCoreStressTest, ClusterStateUpdateTest, TestRandomFaceting, TestZkChroot, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping, TestRecovery, DistributedSpellCheckComponentTest, TestRealTimeGet, TestRangeQuery, TestCoreContainer, TestSolr4Spatial, PeerSyncTest, TestSort, SoftAutoCommitTest, TestTrie, TestCSVLoader, TestUpdate, FieldMutatingUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, SortByFunctionTest, XsltUpdateRequestHandlerTest, TestWriterPerf, TestSolrQueryParser, SolrPluginUtilsTest, ReturnFieldsTest, MBeansHandlerTest, CSVRequestHandlerTest, HighlighterConfigTest, TestQuerySenderNoQuery, BadComponentTest, MultiTermTest, TestBinaryField, NumericFieldsTest, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, EchoParamsTest, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, ResourceLoaderTest, URLClassifyProcessorTest, PreAnalyzedFieldTest, ClusterStateTest, SliceStateTest, FileUtilsTest, CursorPagingTest, CleanupOldIndexTest, ConnectionManagerTest, DeleteInactiveReplicaTest, HttpPartitionTest, SharedFSAutoReplicaFailoverUtilsTest, SolrCloudExampleTest, TestAuthenticationFramework, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestLeaderElectionZkExpiry, TestMiniSolrCloudClusterSSL, TestRandomRequestDistribution, TestReplicaProperties, TriLevelCompositeIdRoutingTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [450/533] on J2 in 361.04s, 1 test, 1 error <<< FAILURES!

[...truncated 267 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 533 suites, 2114 tests, 3 errors, 1 failure, 26 ignored (14 assumptions)

Total time: 196 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 985507060 bytes
Compression is 0.0%
Took 2 min 53 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 929 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/929/

3 tests failed.
REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
Index: 0, Size: 0

Stack Trace:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at __randomizedtesting.SeedInfo.seed([42CF59CE2961C71E:B5BCB796EF8968F8]:0)
	at java.util.ArrayList.rangeCheck(ArrayList.java:635)
	at java.util.ArrayList.get(ArrayList.java:411)
	at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1239)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=54213, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=54213, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:49320: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([42CF59CE2961C71E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:857)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:800)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=1004, name=WRITER3, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1004, name=WRITER3, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
	at __randomizedtesting.SeedInfo.seed([42CF59CE2961C71E:7F4EFD6F7D2A245C]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([42CF59CE2961C71E]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1673)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1530)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:779)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1088)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:705)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
	... 21 more




Build Log:
[...truncated 10205 lines...]
   [junit4] Suite: org.apache.solr.search.TestReloadDeadlock
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001
   [junit4]   2> 104908 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 104909 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 104909 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 104910 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 104910 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 104910 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 104939 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 104953 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 104984 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 104985 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 104998 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 105244 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 105246 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 105247 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 105250 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 105284 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 105285 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 105285 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 105301 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 754356232
   [junit4]   2> 105301 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 105302 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
   [junit4]   2> 105302 WARN  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib).
   [junit4]   2> 105324 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 105326 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 105326 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 105327 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 105327 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 105327 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 105328 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, schema=schema15.xml, instanceDir=collection1, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, shard=shard1, name=collection1, loadOnStartup=true, config=solrconfig-tlog.xml, dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001, collection=collection1}
   [junit4]   2> 105335 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 105336 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 105337 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 105337 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 105363 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 105377 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 105409 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 105409 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 105423 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 105648 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 105649 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 105651 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 105654 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 105682 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 105682 INFO  (coreLoadExecutor-277-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 105683 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 105683 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 105683 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6be19237
   [junit4]   2> 105684 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001
   [junit4]   2> 105684 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001/index/
   [junit4]   2> 105684 WARN  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 105685 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001/index
   [junit4]   2> 105685 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=32, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 105686 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_1,generation=1}
   [junit4]   2> 105686 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 105687 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001/
   [junit4]   2> 105694 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 105694 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 105694 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 105694 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 105695 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 105695 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 105695 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 105696 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 105696 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 105699 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 105700 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 105702 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 105703 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 105708 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/file,standard,/admin/ping,/update/json/docs,/update/csv,/admin/logging,/admin/segments,/config,/admin/mbeans,/admin/luke,/schema,/admin/system,/get,/admin/properties,/update,/admin/plugins,/replication,/update/json
   [junit4]   2> 105709 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 105710 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 105710 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 105711 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 105711 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 105712 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=20.828125, floorSegmentMB=0.541015625, forceMergeDeletesPctAllowed=19.197615364682807, segmentsPerTier=3.0, maxCFSSegmentSizeMB=1.654296875, noCFSRatio=0.8401315866824807
   [junit4]   2> 105712 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_1,generation=1}
   [junit4]   2> 105713 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 105713 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1051694a[collection1] main
   [junit4]   2> 105714 WARN  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 105714 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 105714 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 105714 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 105715 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 105715 INFO  (searcherExecutor-278-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1051694a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 105716 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 105716 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 105716 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 105716 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1509609583937585152
   [junit4]   2> 105718 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1509609583937585152
   [junit4]   2> 105718 INFO  (coreLoadExecutor-277-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 105720 INFO  (SUITE-TestReloadDeadlock-seed#[42CF59CE2961C71E]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 105726 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testReloadDeadlock
   [junit4]   2> 105733 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 105735 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@a0ba648[collection1] realtime
   [junit4]   2> 105735 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=*:* (-9223372036854775807)} 0 7
   [junit4]   2> 105829 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 105830 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_2,generation=2}
   [junit4]   2> 105830 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 105831 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4e1c8a36[collection1] main
   [junit4]   2> 105831 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 105832 INFO  (searcherExecutor-278-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4e1c8a36[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 105834 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 94
   [junit4]   2> 105851 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (2)]} 0 7
   [junit4]   2> 105853 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (5)]} 0 7
   [junit4]   2> 105853 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (6)]} 0 7
   [junit4]   2> 105853 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (1)]} 0 10
   [junit4]   2> 105854 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (4)]} 0 9
   [junit4]   2> 105969 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3c2a25ca[collection1] realtime
   [junit4]   2> 105970 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-7&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-7)} 0 123
   [junit4]   2> 105972 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (8)]} 0 125
   [junit4]   2> 105973 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (9)]} 0 125
   [junit4]   2> 105996 INFO  (WRITER10) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@44c2c269[collection1] realtime
   [junit4]   2> 105998 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-10&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-10)} 0 151
   [junit4]   2> 105999 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (3)]} 0 155
   [junit4]   2> 106002 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4e404fad[collection1] realtime
   [junit4]   2> 106004 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-11&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-11)} 0 156
   [junit4]   2> 106005 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (12)]} 0 156
   [junit4]   2> 106006 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 106007 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 106007 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 106008 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 106015 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3022eec5[collection1] realtime
   [junit4]   2> 106017 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-13&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-13)} 0 165
   [junit4]   2> 106019 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-14&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-14)} 0 164
   [junit4]   2> 106020 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (15)]} 0 165
   [junit4]   2> 106020 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (16)]} 0 166
   [junit4]   2> 106020 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (17)]} 0 166
   [junit4]   2> 106020 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (20)]} 0 47
   [junit4]   2> 106021 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (19)]} 0 48
   [junit4]   2> 106020 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (18)]} 0 50
   [junit4]   2> 106034 INFO  (WRITER10) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5ae568e3[collection1] realtime
   [junit4]   2> 106034 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-21&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-21)} 0 35
   [junit4]   2> 106036 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (22)]} 0 36
   [junit4]   2> 106045 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 106046 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@a7cd08[collection1] realtime
   [junit4]   2> 106047 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-23&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-23)} 0 42
   [junit4]   2> 106049 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@636ad24d[collection1] realtime
   [junit4]   2> 106050 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-24&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-24)} 0 44
   [junit4]   2> 106060 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (25)]} 0 42
   [junit4]   2> 106069 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2302111e[collection1] realtime
   [junit4]   2> 106070 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-26&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-26)} 0 50
   [junit4]   2> 106072 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-27&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-27)} 0 51
   [junit4]   2> 106073 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (30)]} 0 51
   [junit4]   2> 106075 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 106081 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@63d282bf[collection1] realtime
   [junit4]   2> 106082 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-29&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-29)} 0 60
   [junit4]   2> 106083 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (31)]} 0 61
   [junit4]   2> 106104 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3fdd5792[collection1] realtime
   [junit4]   2> 106104 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-28&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-28)} 0 82
   [junit4]   2> 106107 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5e2b3c11[collection1] realtime
   [junit4]   2> 106107 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-32&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-32)} 0 86
   [junit4]   2> 106108 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 106109 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (33)]} 0 74
   [junit4]   2> 106110 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (35)]} 0 59
   [junit4]   2> 106110 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (34)]} 0 62
   [junit4]   2> 106114 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 106116 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 106125 INFO  (WRITER2) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_3,generation=3}
   [junit4]   2> 106126 INFO  (WRITER2) [    ] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 106127 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4fc9584f[collection1] main
   [junit4]   2> 106127 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 106127 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 106128 INFO  (searcherExecutor-278-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4fc9584f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_9(5.4.0):c1) Uninverting(_c(5.4.0):c1) Uninverting(_b(5.4.0):c2/1:delGen=1)))}
   [junit4]   2> 106129 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 92
   [junit4]   2> 106135 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-36&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-36)} 0 75
   [junit4]   2> 106139 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-37&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-37)} 0 68
   [junit4]   2> 106140 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-38&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-38)} 0 68
   [junit4]   2> 106143 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-39&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-39)} 0 69
   [junit4]   2> 106144 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-40&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-40)} 0 61
   [junit4]   2> 106146 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (41)]} 0 61
   [junit4]   2> 106152 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@a3aee42[collection1] realtime
   [junit4]   2> 106153 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-42&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-42)} 0 48
   [junit4]   2> 106154 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (43)]} 0 45
   [junit4]   2> 106154 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[9 (44)]} 0 44
   [junit4]   2> 106167 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@61d93e6a[collection1] realtime
   [junit4]   2> 106168 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-45&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-45)} 0 57
   [junit4]   2> 106169 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-46&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-46)} 0 59
   [junit4]   2> 106171 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2d25fe5e[collection1] realtime
   [junit4]   2> 106171 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-47&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-47)} 0 34
   [junit4]   2> 106173 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (48)]} 0 35
   [junit4]   2> 106173 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (49)]} 0 33
   [junit4]   2> 106173 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (50)]} 0 32
   [junit4]   2> 106185 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6ab494c6[collection1] realtime
   [junit4]   2> 106199 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-52&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-52)} 0 54
   [junit4]   2> 106199 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-51&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-51)} 0 55
   [junit4]   2> 106201 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (53)]} 0 54
   [junit4]   2> 106215 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6aed097d[collection1] realtime
   [junit4]   2> 106218 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (55)]} 0 62
   [junit4]   2> 106218 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (56)]} 0 63
   [junit4]   2> 106218 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (57)]} 0 49
   [junit4]   2> 106218 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-54&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-54)} 0 64
   [junit4]   2> 106230 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6f3c1f5c[collection1] realtime
   [junit4]   2> 106231 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-58&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-58)} 0 61
   [junit4]   2> 106233 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-59&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-59)} 0 61
   [junit4]   2> 106235 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (60)]} 0 62
   [junit4]   2> 106236 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (62)]} 0 62
   [junit4]   2> 106236 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (61)]} 0 62
   [junit4]   2> 106246 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@26a006f4[collection1] realtime
   [junit4]   2> 106247 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-63&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-63)} 0 46
   [junit4]   2> 106249 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[9 (64)]} 0 48
   [junit4]   2> 106249 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (65)]} 0 48
   [junit4]   2> 106250 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (67)]} 0 31
   [junit4]   2> 106250 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (66)]} 0 31
   [junit4]   2> 106251 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (68)]} 0 32
   [junit4]   2> 106272 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@68bd1f7d[collection1] realtime
   [junit4]   2> 106274 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-69&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-69)} 0 54
   [junit4]   2> 106284 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-70&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-70)} 0 52
   [junit4]   2> 106286 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (71)]} 0 51
   [junit4]   2> 106296 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@43520332[collection1] realtime
   [junit4]   2> 106299 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-72&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-72)} 0 64
   [junit4]   2> 106302 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-73&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-73)} 0 66
   [junit4]   2> 106304 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (74)]} 0 67
   [junit4]   2> 106331 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4f342093[collection1] realtime
   [junit4]   2> 106332 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-76&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-76)} 0 82
   [junit4]   2> 106338 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (75)]} 0 90
   [junit4]   2> 106341 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-77&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-77)} 0 92
   [junit4]   2> 106343 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (79)]} 0 92
   [junit4]   2> 106343 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (78)]} 0 91
   [junit4]   2> 106352 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@598ceaa3[collection1] realtime
   [junit4]   2> 106354 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (81)]} 0 80
   [junit4]   2> 106374 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1b606b8c[collection1] realtime
   [junit4]   2> 106375 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-80&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-80)} 0 123
   [junit4]   2> 106379 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-83&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-83)} 0 92
   [junit4]   2> 106381 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (84)]} 0 80
   [junit4]   2> 106381 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-82&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-82)} 0 95
   [junit4]   2> 106389 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4a720218[collection1] realtime
   [junit4]   2> 106394 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5fffa9a9[collection1] realtime
   [junit4]   2> 106394 INFO  (WRITER9) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 106397 INFO  (WRITER9) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_3,generation=3}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_4,generation=4}
   [junit4]   2> 106397 INFO  (WRITER9) [    ] o.a.s.c.SolrCore newest commit generation = 4
   [junit4]   2> 106398 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-86&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-86)} 0 92
   [junit4]   2> 106405 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-85&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-85)} 0 101
   [junit4]   2> 106406 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@53bc0bb2[collection1] main
   [junit4]   2> 106414 INFO  (searcherExecutor-278-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@53bc0bb2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_9(5.4.0):c1) Uninverting(_l(5.4.0):c2/1:delGen=1) Uninverting(_o(5.4.0):c1) Uninverting(_m(5.4.0):c1) Uninverting(_p(5.4.0):c1) Uninverting(_t(5.4.0):c1) Uninverting(_v(5.4.0):c1)))}
   [junit4]   2> 106415 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (87)]} 0 77
   [junit4]   2> 106416 INFO  (WRITER9) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 106419 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 86
   [junit4]   2> 106423 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-88&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-88)} 0 81
   [junit4]   2> 106424 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (89)]} 0 80
   [junit4]   2> 106433 INFO  (WRITER10) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6498a2b8[collection1] realtime
   [junit4]   2> 106437 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@69dad81f[collection1] realtime
   [junit4]   2> 106440 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (94)]} 0 59
   [junit4]   2> 106441 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (95)]} 0 59
   [junit4]   2> 106442 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (93)]} 0 62
   [junit4]   2> 106442 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-91&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-91)} 0 86
   [junit4]   2> 106445 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (92)]} 0 69
   [junit4]   2> 106447 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (96)]} 0 48
   [junit4]   2> 106449 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-90&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-90)} 0 104
   [junit4]   2> 106452 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (100)]} 0 27
   [junit4]   2> 106452 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (99)]} 0 28
   [junit4]   2> 106453 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (98)]} 0 37
   [junit4]   2> 106476 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 106478 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5f729832[collection1] realtime
   [junit4]   2> 106478 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 106479 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (97)]} 0 73
   [junit4]   2> 106479 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 106480 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (102)]} 0 39
   [junit4]   2> 106483 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 106493 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@17537fc7[collection1] realtime
   [junit4]   2> 106495 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-104&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-104)} 0 52
   [junit4]   2> 106498 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-105&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-105)} 0 52
   [junit4]   2> 106501 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-106&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-106)} 0 53
   [junit4]   2> 106504 INFO  (WRITER10) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@20eca2cd[collection1] realtime
   [junit4]   2> 106506 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-109&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-109)} 0 54
   [junit4]   2> 106507 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-108&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-108)} 0 57
   [junit4]   2> 106507 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-103&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-103)} 0 65
   [junit4]   2> 106508 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (110)]} 0 54
   [junit4]   2> 106508 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-101&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-101)} 0 67
   [junit4]   2> 106516 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 106516 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 106516 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6eb4b976[collection1] realtime
   [junit4]   2> 106516 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001/]
   [junit4]   2> 106517 INFO  (WRITER9) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=107,id=0} DBQs=[DBQ{version=111,q=id:9}, DBQ{version=109,q=id:1}, DBQ{version=108,q=id:1}]
   [junit4]   2> 106517 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-111&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-111)} 0 63
   [junit4]   2> 106518 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6be19237
   [junit4]   2> 106520 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001/index/
   [junit4]   2> 106534 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_42CF59CE2961C71E-001/init-core-data-001/
   [junit4]   2> 106536 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2d75dfe0[collection1] realtime
   [junit4]   2> 106537 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (107)]} 0 83
   [junit4]   2> 106537 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 106538 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 106538 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 106538 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 106538 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 106539 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 106539 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 106539 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 106540 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 106541 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@62b6d61b[collection1] realtime
   [junit4]   2> 106541 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-112&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-112)} 0 62
   [junit4]   2> 106543 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 106545 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5c939dd0[collection1] realtime
   [junit4]   2> 106545 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 106547 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 106548 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@47350201[collection1] realtime
   [junit4]   2> 106549 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 106550 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (115)]} 0 50
   [junit4]   2> 106550 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (116)]} 0 49
   [junit4]   2> 106551 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (119)]} 0 43
   [junit4]   2> 106553 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-113&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-113)} 0 72
   [junit4]   2> 106554 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (118)]} 0 46
   [junit4]   2> 106556 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/file,standard,/admin/ping,/update/json/docs,/update/csv,/admin/logging,/admin/segments,/config,/admin/mbeans,/admin/luke,/schema,/admin/system,/get,/admin/properties,/update,/admin/plugins,/replication,/update/json
   [junit4]   2> 106558 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 106558 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 106559 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 106616 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1d80e5fb[collection1] realtime
   [junit4]   2> 106617 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-114&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-114)} 0 120
   [junit4]   2> 106618 ERROR (WRITER2) [    ] o.a.s.u.UpdateLog Error opening realtime searcher for deleteByQuery:org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1673)
   [junit4]   2> 	at org.apache.solr.update.UpdateLog.deleteByQuery(UpdateLog.java:521)
   [junit4]   2> 	at org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:450)
   [junit4]   2> 	at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:80)
   [junit4]   2> 	at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:55)
   [junit4]   2> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:959)
   [junit4]   2> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.doDeleteByQuery(DistributedUpdateProcessor.java:1371)
   [junit4]   2> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1153)
   [junit4]   2> 	at org.apache.solr.update.processor.LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:125)
   [junit4]   2> 	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleDeleteMap(JsonLoader.java:366)
   [junit4]   2> 	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleDeleteCommand(JsonLoader.java:283)
   [junit4]   2> 	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:160)
   [junit4]   2> 	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
   [junit4]   2> 	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
   [junit4]   2> 	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
   [junit4]   2> 	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.deleteByQueryAndGetVersion(SolrTestCaseJ4.java:1276)
   [junit4]   2> 	at org.apache.solr.search.TestReloadDeadlock.deleteByQuery(TestReloadDeadlock.java:220)
   [junit4]   2> 	at org.apache.solr.search.TestReloadDeadlock.access$000(TestReloadDeadlock.java:46)
   [junit4]   2> 	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:154)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
   [junit4]   2> 	... 23 more
   [junit4]   2> 
   [junit4]   2> 106619 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (117)]} 0 111
   [junit4]   2> 106623 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2f813e50[collection1] realtime
   [junit4]   2> 106623 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@77f13049[collection1] main
   [junit4]   2> 106624 WARN  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 106624 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 106624 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 106625 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 106625 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 106625 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 106626 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4a135c46[collection1] realtime
   [junit4]   2> 106626 INFO  (searcherExecutor-283-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@77f13049[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_9(5.4.0):c1) Uninverting(_o(5.4.0):c1) Uninverting(_y(5.4.0):c5/4) Uninverting(_12(5.4.0):c5/2:delGen=1)))}
   [junit4]   2> 106627 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 106627 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-123&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:2 (-123)} 0 89
   [junit4]   2> 106634 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_4,generation=4}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_5,generation=5}
   [junit4]   2> 106634 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 5
   [junit4]   2> 106635 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (126)]} 0 82
   [junit4]   2> 106636 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-122&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-122)} 0 117
   [junit4]   2> 106636 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-121&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-121)} 0 127
   [junit4]   2> 106636 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-120&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-120)} 0 128
   [junit4]   2> 106641 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@21c359a0[collection1] main
   [junit4]   2> 106642 INFO  (searcherExecutor-278-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@21c359a0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_o(5.4.0):c1) Uninverting(_y(5.4.0):c5/4:delGen=1) Uninverting(_12(5.4.0):c5/3:delGen=2) Uninverting(_13(5.4.0):c1)))}
   [junit4]   2> 106644 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 106645 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 102
   [junit4]   2> 106645 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (124)]} 0 92
   [junit4]   2> 106653 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@f1673f2[collection1] realtime
   [junit4]   2> 106656 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (128)]} 0 100
   [junit4]   2> 106657 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (129)]} 0 39
   [junit4]   2> 106657 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (130)]} 0 38
   [junit4]   2> 106658 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (127)]} 0 103
   [junit4]   2> 106659 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-125&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-125)} 0 106
   [junit4]   2> 106659 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 106684 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.VersionInfo Found MAX value 119 from Terms for _version_ in index
   [junit4]   2> 106686 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.UpdateLog Took 28.0ms to seed version buckets with highest version 130
   [junit4]   2> 106694 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Creating new IndexWriter...
   [junit4]   2> 106694 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Waiting until IndexWriter is unused... core=collection1
   [junit4]   2> 106704 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Closing old IndexWriter... core=collection1
   [junit4]   2> 106704 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@37b211c2[collection1] realtime
   [junit4]   2> 106707 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-131&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-131)} 0 73
   [junit4]   2> 106708 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_5,generation=5}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_6,generation=6}
   [junit4]   2> 106708 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 6
   [junit4]   2> 106710 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=2105773112, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 106720 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12826436 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e811eb8),segFN=segments_6,generation=6}
   [junit4]   2> 106726 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 6
   [junit4]   2> 106727 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 106740 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6e76b039[collection1] main
   [junit4]   2> 106741 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.CoreContainer replacing core: collection1
   [junit4]   2> 106742 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5ab4cfb2
   [junit4]   2> 106741 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-133&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-133)} 0 105
   [junit4]   2> 106742 INFO  (searcherExecutor-283-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6e76b039[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_y(5.4.0):c5/4:delGen=1) Uninverting(_12(5.4.0):c5/4:delGen=3) Uninverting(_13(5.4.0):c1) Uninverting(_14(5.4.0):c1) Uninverting(_15(5.4.0):c1) Uninverting(_16(5.4.0):c1)))}
   [junit4]   2> 106746 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=6,adds=4,deletesById=0,deletesByQuery=4,errors=1,cumulative_adds=65,cumulative_deletesById=0,cumulative_deletesByQuery=62,cumulative_errors=1,transaction_logs_total_size=3179,transaction_logs_total_number=3}
   [junit4]   2> 106746 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[42CF59CE2961C71E]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 106749 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-132&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-132)} 0 112
   [junit4]   2> 106751 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (134)]} 0 114
   [junit4]   2> 106752 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (136)]} 0 106
   [junit4]   2> 106754 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (137)]} 0 105
   [junit4]   2> 106755 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (135)]} 0 118
   [junit4]   2> 106780 INFO  (WRITER11) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1d801cf2[collection1] realtime
   [junit4]   2> 106783 INFO  (WRITER10) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (141)]} 0 124
   [junit4]   2> 106785 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (139)]} 0 126
   [junit4]   2> 106786 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (140)]} 0 127
   [junit4]   2> 106786 INFO  (WRITER11) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-138&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-138)} 0 129
   [junit4]   2> 106803 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@51e4bccc[collection1] realtime
   [junit4]   2> 106810 INFO  (WRITER9) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@43768c6c[collection1] realtime
   [junit4]   2> 106814 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-144&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-144)} 0 70
   [junit4]   2> 106818 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1db0e06a[collection1] realtime
   [junit4]   2> 106819 INFO  (WRITER9) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-143&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-143)} 0 111
   [junit4]   2> 106820 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (147)]} 0 67
   [junit4]   2> 106820 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (148)]} 0 64
   [junit4]   2> 106820 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-142&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-142)} 0 160
   [junit4]   2> 106828 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@19a26a1[collection1] realtime
   [junit4]   2> 106829 INFO  (WRITER1) [    ] o

[...truncated too long message...]

MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 1591304 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[42CF59CE2961C71E]-worker) [n:127.0.0.1:47223_ c:addReplicaColl s:shard2 r:core_node2 x:addReplicaColl_shard2_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 15, 2015 6:22:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=21012, maxDocsPerChunk=2, blockSize=375), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=21012, blockSize=375)), sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=lt_LT, timezone=Asia/Dhaka
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=216393392,total=475004928
   [junit4]   2> NOTE: All tests run in this JVM: [SpatialHeatmapFacetsTest, CircularListTest, CoreMergeIndexesAdminHandlerTest, TestUtils, TestSolrQueryParserResource, DeleteShardTest, DistribJoinFromCollectionTest, SolrInfoMBeanTest, HdfsDirectoryTest, SegmentsInfoRequestHandlerTest, BlockCacheTest, DOMUtilTest, UnloadDistributedZkTest, QueryParsingTest, TestDocSet, TestDocBasedVersionConstraints, HttpPartitionTest, AnalysisErrorHandlingTest, TestSolrQueryParserDefaultOperatorResource, MultiThreadedOCPTest, TestBinaryResponseWriter, CoreAdminCreateDiscoverTest, TestTrie, SOLR749Test, BinaryUpdateRequestHandlerTest, SynonymTokenizerTest, QueryResultKeyTest, TestDistribIDF, ResponseLogComponentTest, TestSmileRequest, TestStressReorder, TestRebalanceLeaders, TestPerFieldSimilarity, StressHdfsTest, BufferStoreTest, TestAnalyzeInfixSuggestions, OverseerTest, CollectionReloadTest, AddSchemaFieldsUpdateProcessorFactoryTest, DistributedQueryComponentCustomSortTest, TestExceedMaxTermLength, SampleTest, PathHierarchyTokenizerFactoryTest, CoreAdminRequestStatusTest, TestPseudoReturnFields, HdfsUnloadDistributedZkTest, DirectUpdateHandlerOptimizeTest, TestCloudInspectUtil, TestSchemaManager, VersionInfoTest, TestJettySolrRunner, MoreLikeThisHandlerTest, SuggesterTest, AddBlockUpdateTest, TestXIncludeConfig, TestFieldTypeResource, PolyFieldTest, TestGroupingSearch, TestConfigSets, NumericFieldsTest, BadIndexSchemaTest, CursorMarkTest, TestReloadAndDeleteDocs, SimpleFacetsTest, BigEndianAscendingWordSerializerTest, TestRestoreCore, TestPartialUpdateDeduplication, TestRandomRequestDistribution, DistributedExpandComponentTest, PrimUtilsTest, SolrIndexSplitterTest, SpellCheckCollatorTest, TestFastOutputStream, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [207/533] on J2 in 348.08s, 1 test, 1 error <<< FAILURES!

[...truncated 1065 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 533 suites, 2114 tests, 3 errors, 26 ignored (14 assumptions)

Total time: 199 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1185534510 bytes
Compression is 0.0%
Took 3 min 16 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 928 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/928/

3 tests failed.
REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1439563750579,generation=2,filelist=[_1rd.cfe, _1rd.cfs, _1rd.si, _1zq.cfe, _1zq.cfs, _1zq.si, _1zr.cfe, _1zr.cfs, _1zr.si, _1zs.cfe, _1zs.cfs, _1zs.si, _vo.fdt, _vo.fdx, _vo.fnm, _vo.nvd, _vo.nvm, _vo.si, _vo_FSTOrd50_0.doc, _vo_FSTOrd50_0.tbk, _vo_FSTOrd50_0.tix, segments_2]}]> but was:<[{indexVersion=1439563750579,generation=3,filelist=[_1rd.cfe, _1rd.cfs, _1rd.si, _1zt.cfe, _1zt.cfs, _1zt.si, _vo.fdt, _vo.fdx, _vo.fnm, _vo.nvd, _vo.nvm, _vo.si, _vo_FSTOrd50_0.doc, _vo_FSTOrd50_0.tbk, _vo_FSTOrd50_0.tix, segments_3]}, {indexVersion=1439563750579,generation=2,filelist=[_1rd.cfe, _1rd.cfs, _1rd.si, _1zq.cfe, _1zq.cfs, _1zq.si, _1zr.cfe, _1zr.cfs, _1zr.si, _1zs.cfe, _1zs.cfs, _1zs.si, _vo.fdt, _vo.fdx, _vo.fnm, _vo.nvd, _vo.nvm, _vo.si, _vo_FSTOrd50_0.doc, _vo_FSTOrd50_0.tbk, _vo_FSTOrd50_0.tix, segments_2]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1439563750579,generation=2,filelist=[_1rd.cfe, _1rd.cfs, _1rd.si, _1zq.cfe, _1zq.cfs, _1zq.si, _1zr.cfe, _1zr.cfs, _1zr.si, _1zs.cfe, _1zs.cfs, _1zs.si, _vo.fdt, _vo.fdx, _vo.fnm, _vo.nvd, _vo.nvm, _vo.si, _vo_FSTOrd50_0.doc, _vo_FSTOrd50_0.tbk, _vo_FSTOrd50_0.tix, segments_2]}]> but was:<[{indexVersion=1439563750579,generation=3,filelist=[_1rd.cfe, _1rd.cfs, _1rd.si, _1zt.cfe, _1zt.cfs, _1zt.si, _vo.fdt, _vo.fdx, _vo.fnm, _vo.nvd, _vo.nvm, _vo.si, _vo_FSTOrd50_0.doc, _vo_FSTOrd50_0.tbk, _vo_FSTOrd50_0.tix, segments_3]}, {indexVersion=1439563750579,generation=2,filelist=[_1rd.cfe, _1rd.cfs, _1rd.si, _1zq.cfe, _1zq.cfs, _1zq.si, _1zr.cfe, _1zr.cfs, _1zr.si, _1zs.cfe, _1zs.cfs, _1zs.si, _vo.fdt, _vo.fdx, _vo.fnm, _vo.nvd, _vo.nvm, _vo.si, _vo_FSTOrd50_0.doc, _vo_FSTOrd50_0.tbk, _vo_FSTOrd50_0.tix, segments_2]}]>
	at __randomizedtesting.SeedInfo.seed([F97A676F4346D42:2A40BD46847C6341]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=16358, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16358, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43518/ta: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([F97A676F4346D42]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=239329, name=WRITER11, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=239329, name=WRITER11, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
	at __randomizedtesting.SeedInfo.seed([F97A676F4346D42:321602D7A07F8E00]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([F97A676F4346D42]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1662)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1519)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:778)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1089)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1631)
	... 21 more




Build Log:
[...truncated 10924 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/init-core-data-001
   [junit4]   2> 936495 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 936495 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ta/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 936529 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 936536 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 936539 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 936551 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_32818_hdfs____.9qks6/webapp
   [junit4]   2> 936727 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 937278 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32818
   [junit4]   2> 937896 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 937898 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 937913 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33914_datanode____jzhbc3/webapp
   [junit4]   2> 938030 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 938334 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33914
   [junit4]   2> 938475 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 938478 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 938490 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40108_datanode____.qw4v0/webapp
   [junit4]   2> 938525 INFO  (IPC Server handler 3 on 49717) [    ] BlockStateChange BLOCK* processReport: from storage DS-443182e1-cffc-4fca-8f98-37059a039cfa node DatanodeRegistration(127.0.0.1, datanodeUuid=94ceb2ab-e9dd-419f-b985-df88cfc82c00, infoPort=33914, ipcPort=54235, storageInfo=lv=-56;cid=testClusterID;nsid=873355876;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 938526 INFO  (IPC Server handler 3 on 49717) [    ] BlockStateChange BLOCK* processReport: from storage DS-99ab16a4-bd15-43f7-9a2f-939bee883c0b node DatanodeRegistration(127.0.0.1, datanodeUuid=94ceb2ab-e9dd-419f-b985-df88cfc82c00, infoPort=33914, ipcPort=54235, storageInfo=lv=-56;cid=testClusterID;nsid=873355876;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 938698 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 940116 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F97A676F4346D42]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40108
   [junit4]   2> 940485 INFO  (IPC Server handler 7 on 49717) [    ] BlockStateChange BLOCK* processReport: from storage DS-820996c1-2393-4960-b196-5a365d7e29aa node DatanodeRegistration(127.0.0.1, datanodeUuid=afff404c-edac-421f-afdf-e96aa47dcd3c, infoPort=40108, ipcPort=50443, storageInfo=lv=-56;cid=testClusterID;nsid=873355876;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 940485 INFO  (IPC Server handler 7 on 49717) [    ] BlockStateChange BLOCK* processReport: from storage DS-dd02a8d0-7c14-453c-bc52-545580d37545 node DatanodeRegistration(127.0.0.1, datanodeUuid=afff404c-edac-421f-afdf-e96aa47dcd3c, infoPort=40108, ipcPort=50443, storageInfo=lv=-56;cid=testClusterID;nsid=873355876;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 940555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 940566 INFO  (Thread-11574) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 940566 INFO  (Thread-11574) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 940666 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.ZkTestServer start zk server on port:45420
   [junit4]   2> 940666 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 940689 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 940713 INFO  (zkCallback-654-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bb847e2 name:ZooKeeperConnection Watcher:127.0.0.1:45420 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 940713 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 940714 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 940714 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 940716 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 940762 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 940773 INFO  (zkCallback-655-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7cb51fdd name:ZooKeeperConnection Watcher:127.0.0.1:45420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 940773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 940774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 940774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 940775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 940777 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 940778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 940779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 940779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 940781 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 940782 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 940783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 940784 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 940785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 940785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 940787 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 940787 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 940788 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 940788 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 940790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 940790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 940791 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 940792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 940793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 940793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 940794 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 940795 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 940796 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 940796 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 940798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 940817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 940824 INFO  (zkCallback-656-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c34a2ea name:ZooKeeperConnection Watcher:127.0.0.1:45420 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 940824 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 940824 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 940824 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 940826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 940830 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 940832 INFO  (zkCallback-657-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d29d0ce name:ZooKeeperConnection Watcher:127.0.0.1:45420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 940832 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 940832 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 940833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 940833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 940849 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 940849 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 940851 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 940851 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 940853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 940853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 940855 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 940855 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 940857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 940857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 940860 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 940860 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 940865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 940865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 940867 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 940867 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 940878 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 940878 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 940879 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 940880 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 940881 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 940881 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 941415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores/collection1
   [junit4]   2> 941416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 941418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@109bc3c1{/ta,null,AVAILABLE}
   [junit4]   2> 941419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2282473a{HTTP/1.1}{127.0.0.1:52915}
   [junit4]   2> 941419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.Server Started @944410ms
   [junit4]   2> 941419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:49717/hdfs__localhost_49717__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001_tempDir-002_control_data, hostPort=52915, hostContext=/ta, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores}
   [junit4]   2> 941419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 941419 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/'
   [junit4]   2> 941434 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 941435 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 941437 INFO  (zkCallback-658-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73c54b4e name:ZooKeeperConnection Watcher:127.0.0.1:45420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 941437 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 941437 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 941439 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 941439 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/solr.xml
   [junit4]   2> 941452 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores
   [junit4]   2> 941453 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.CoreContainer New CoreContainer 1204589
   [junit4]   2> 941453 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/]
   [junit4]   2> 941453 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/lib
   [junit4]   2> 941453 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/lib).
   [junit4]   2> 941461 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 941462 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 941463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 941463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 941463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45420/solr
   [junit4]   2> 941463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 941463 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 941466 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 941466 INFO  (zkCallback-660-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@52323802 name:ZooKeeperConnection Watcher:127.0.0.1:45420 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 941467 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 941467 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 941473 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 941473 INFO  (zkCallback-661-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c6e1fdf name:ZooKeeperConnection Watcher:127.0.0.1:45420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 941473 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 941475 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 941476 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 941478 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 941479 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 941481 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 941482 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 941483 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 941484 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 941485 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 941486 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52915_ta
   [junit4]   2> 941487 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52915_ta
   [junit4]   2> 941488 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 941489 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 941490 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 941490 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94343051074207750-127.0.0.1:52915_ta-n_0000000000
   [junit4]   2> 941491 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52915_ta
   [junit4]   2> 941491 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 941492 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.Overseer Overseer (id=94343051074207750-127.0.0.1:52915_ta-n_0000000000) starting
   [junit4]   2> 941493 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 941498 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 941499 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 941499 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 941502 INFO  (OverseerCollectionProcessor-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.OverseerProcessor Process current queue of overseer operations
   [junit4]   2> 941504 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 941522 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 941523 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 941523 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores
   [junit4]   2> 941524 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, shard=, configSetProperties=configsetprops.json, collection=control_collection, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores/collection1, dataDir=data/, coreNodeName=, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores/collection1/, transient=false, loadOnStartup=true}
   [junit4]   2> 941527 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores/collection1/
   [junit4]   2> 941527 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 941542 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 941542 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 941543 INFO  (zkCallback-661-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 941543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 941543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [n:127.0.0.1:52915_ta    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 941544 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 941544 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52915/ta",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:52915_ta"} current state version: 0
   [junit4]   2> 941544 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52915/ta",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:52915_ta"}
   [junit4]   2> 941545 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 941545 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 941546 INFO  (zkCallback-661-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 941549 INFO  (zkCallback-661-thread-2-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 942544 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 942544 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 942545 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 942545 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 942545 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 942545 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores/collection1/'
   [junit4]   2> 942574 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 942580 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 942592 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 942658 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 942660 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 942678 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 943186 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 943194 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 943196 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 943206 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 943209 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 943211 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 943212 WARN  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 943212 WARN  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 943212 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 943213 WARN  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 943213 WARN  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 943214 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 943214 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 943214 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 943215 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49717/solr_hdfs_home
   [junit4]   2> 943215 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 943215 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 943215 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e306231
   [junit4]   2> 943215 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 943254 WARN  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 948257 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 948258 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 948258 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 948269 WARN  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:49717/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 948269 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 948283 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 948283 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 948283 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1038925330, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 948299 INFO  (IPC Server handler 6 on 49717) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57828 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-dd02a8d0-7c14-453c-bc52-545580d37545:NORMAL:127.0.0.1:53031|RBW], ReplicaUnderConstruction[[DISK]DS-443182e1-cffc-4fca-8f98-37059a039cfa:NORMAL:127.0.0.1:57828|FINALIZED]]} size 0
   [junit4]   2> 948300 INFO  (IPC Server handler 5 on 49717) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:53031 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-dd02a8d0-7c14-453c-bc52-545580d37545:NORMAL:127.0.0.1:53031|RBW], ReplicaUnderConstruction[[DISK]DS-443182e1-cffc-4fca-8f98-37059a039cfa:NORMAL:127.0.0.1:57828|FINALIZED]]} size 0
   [junit4]   2> 948304 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7efddf7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33d0c0d1; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 948304 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 948333 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 948333 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 948333 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 948333 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 948333 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 948333 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 948334 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 948334 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 948334 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 948337 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 948338 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 948339 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 948341 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:49717/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 948349 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 948352 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,standard,/admin/system,/admin/plugins,/get,/update,/admin/ping,/update/csv,/admin/segments,/config,/replication,/admin/properties,/admin/logging,/admin/threads,/schema,/admin/file,/admin/mbeans,/update/json/docs,/admin/luke
   [junit4]   2> 948354 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 948361 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 948361 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 948395 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 948395 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 948396 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=1.0888671875, floorSegmentMB=1.8076171875, forceMergeDeletesPctAllowed=16.905176966178953, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.44667804740121575
   [junit4]   2> 948419 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7efddf7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33d0c0d1; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 948420 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 948420 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3feae8b7[collection1] main
   [junit4]   2> 948420 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 948421 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 948421 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 948421 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 948421 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 948422 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 948422 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 948422 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 948422 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 948422 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 948423 INFO  (searcherExecutor-2846-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3feae8b7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 948425 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 948426 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 948426 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 948426 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1509488825428082688
   [junit4]   2> 948426 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0 ms to seed version buckets with highest version 1509488825428082688
   [junit4]   2> 948426 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 948427 INFO  (coreLoadExecutor-2845-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 948438 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:52915/ta collection:control_collection shard:shard1
   [junit4]   2> 948441 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 948445 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94343051074207750-core_node1-n_0000000000
   [junit4]   2> 948446 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 948447 INFO  (zkCallback-661-thread-2-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 948447 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 948448 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 948448 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52915/ta/collection1/
   [junit4]   2> 948448 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 948449 INFO  (zkCallback-661-thread-2-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948448 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 948449 INFO  (zkCallback-661-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948451 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52915/ta/collection1/ has no replicas
   [junit4]   2> 948453 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52915/ta/collection1/ shard1
   [junit4]   2> 948453 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 948457 INFO  (zkCallback-661-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 948457 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:52915/ta",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 948546 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 948560 INFO  (zkCallback-661-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948560 INFO  (zkCallback-661-thread-2-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948569 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 948585 INFO  (zkCallback-663-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25361c97 name:ZooKeeperConnection Watcher:127.0.0.1:45420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 948585 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 948586 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 948586 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 948588 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 948609 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:52915/ta/collection1/ and leader is http://127.0.0.1:52915/ta/collection1/
   [junit4]   2> 948609 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52915/ta
   [junit4]   2> 948609 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 948610 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 948610 INFO  (coreZkRegister-2839-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 948611 INFO  (zkCallback-661-thread-2-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 948611 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52915/ta",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:52915_ta"} current state version: 3
   [junit4]   2> 948612 INFO  (OverseerStateUpdate-94343051074207750-127.0.0.1:52915_ta-n_0000000000) [n:127.0.0.1:52915_ta    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52915/ta",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:52915_ta"}
   [junit4]   2> 948716 INFO  (zkCallback-663-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948716 INFO  (zkCallback-661-thread-2-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948717 INFO  (zkCallback-661-thread-1-processing-n:127.0.0.1:52915_ta) [n:127.0.0.1:52915_ta    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948931 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/shard-1-001/cores/collection1
   [junit4]   2> 948932 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/shard-1-001
   [junit4]   2> 948933 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 948941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d510b8a{/ta,null,AVAILABLE}
   [junit4]   2> 948942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.ServerConnector Started ServerConnector@33cca00f{HTTP/1.1}{127.0.0.1:46514}
   [junit4]   2> 948942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.e.j.s.Server Started @951933ms
   [junit4]   2> 948942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:49717/hdfs__localhost_49717__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001_tempDir-002_jetty1, hostContext=/ta, hostPort=46514, solrconfig=solrconfig.xml}
   [junit4]   2> 948942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 948943 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F97A676F4346D42-001/shard-1-001/'
   [junit4]   2> 948962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 948981 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F97A676F4346D42]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 948991 INFO  (zkCallback-664-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24aedbd3 name:ZooKeeperConnection Watcher:127.0.0.1:45420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4] 

[...truncated too long message...]

e/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150814_094252_144.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150814_094252_144.suites
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 218 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1403298715 bytes
Compression is 0.0%
Took 4 min 45 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 927 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/927/

6 tests failed.
REGRESSION:  org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([6F74601963E2F341:5ECFDE2CC6DDE391]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:765)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:732)
	at org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts(SpellCheckCollatorTest.java:530)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='spellcheck']/lst[@name='collations']/lst[@name='collation']/int[@name='hits' and 6 <= . and . <= 10]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">3</int></lst><result name="response" numFound="0" start="0"></result><lst name="spellcheck"><lst name="suggestions"><lst name="everother"><int name="numFound">1</int><int name="startOffset">9</int><int name="endOffset">18</int><arr name="suggestion"><str>everyother</str></arr></lst></lst><lst name="collations"><lst name="collation"><str name="collationQuery">teststop:everyother</str><int name="hits">14</int><lst name="misspellingsAndCorrections"><str name="everother">everyother</str></lst></lst></lst></lst>
</response>

	request was:spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Aeverother&spellcheck.collateMaxCollectDocs=5
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:758)
	... 41 more


FAILED:  org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail

Error Message:
concurrent create and delete collection failed: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41928/solr: Could not fully remove collection: collection8

Stack Trace:
java.lang.AssertionError: concurrent create and delete collection failed: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41928/solr: Could not fully remove collection: collection8
	at __randomizedtesting.SeedInfo.seed([6F74601963E2F341:AE5D35A314392950]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail(ConcurrentDeleteAndCreateCollectionTest.java:72)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=23365, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=23365, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40579: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([6F74601963E2F341]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12103, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12103, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47968/_aia/q: collection already exists: awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([6F74601963E2F341]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=249595, name=WRITER0, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=249595, name=WRITER0, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
	at __randomizedtesting.SeedInfo.seed([6F74601963E2F341:52F5C4B837A91003]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([6F74601963E2F341]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1662)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1519)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:778)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1089)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1631)
	... 21 more


FAILED:  org.apache.solr.util.TestSolrCLIRunExample.testTechproductsExample

Error Message:
expected 32 docs in the techproducts example but found 0, output: Creating Solr home directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.util.TestSolrCLIRunExample_6F74601963E2F341-001/tempDir-001/techproducts/solr  Starting up Solr on port 44038 using command: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/bin/solr start -p 44038 -s "temp/solr.util.TestSolrCLIRunExample_6F74601963E2F341-001/tempDir-001/techproducts/solr"   Setup new core instance directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.util.TestSolrCLIRunExample_6F74601963E2F341-001/tempDir-001/techproducts/solr/techproducts  Creating new core 'techproducts' using command: http://localhost:44038/solr/admin/cores?action=CREATE&name=techproducts&instanceDir=techproducts  {   "responseHeader":{     "status":0,     "QTime":671},   "core":"techproducts"}   Indexing tech product example docs from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/exampledocs  Solr techproducts example launched successfully. Direct your Web browser to http://localhost:44038/solr to visit the Solr Admin UI 

Stack Trace:
java.lang.AssertionError: expected 32 docs in the techproducts example but found 0, output: Creating Solr home directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.util.TestSolrCLIRunExample_6F74601963E2F341-001/tempDir-001/techproducts/solr

Starting up Solr on port 44038 using command:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/bin/solr start -p 44038 -s "temp/solr.util.TestSolrCLIRunExample_6F74601963E2F341-001/tempDir-001/techproducts/solr"


Setup new core instance directory:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.util.TestSolrCLIRunExample_6F74601963E2F341-001/tempDir-001/techproducts/solr/techproducts

Creating new core 'techproducts' using command:
http://localhost:44038/solr/admin/cores?action=CREATE&name=techproducts&instanceDir=techproducts

{
  "responseHeader":{
    "status":0,
    "QTime":671},
  "core":"techproducts"}


Indexing tech product example docs from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/exampledocs

Solr techproducts example launched successfully. Direct your Web browser to http://localhost:44038/solr to visit the Solr Admin UI

	at __randomizedtesting.SeedInfo.seed([6F74601963E2F341:E5A3B0A17900CBDE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.util.TestSolrCLIRunExample.testExample(TestSolrCLIRunExample.java:347)
	at org.apache.solr.util.TestSolrCLIRunExample.testTechproductsExample(TestSolrCLIRunExample.java:284)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10677 lines...]
   [junit4] Suite: org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/init-core-data-001
   [junit4]   2> 721382 INFO  (SUITE-ConcurrentDeleteAndCreateCollectionTest-seed#[6F74601963E2F341]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 721387 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testConcurrentCreateAndDeleteDoesNotFail
   [junit4]   2> 721388 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 721389 INFO  (Thread-46760) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 721389 INFO  (Thread-46760) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 721489 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.ZkTestServer start zk server on port:46808
   [junit4]   2> 721489 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 721500 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 721506 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a8dc32c name:ZooKeeperConnection Watcher:127.0.0.1:46808 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721506 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 721506 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 721507 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 721536 INFO  (jetty-launcher-170-thread-1) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 721572 INFO  (jetty-launcher-170-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7cd097e2{/solr,null,AVAILABLE}
   [junit4]   2> 721574 INFO  (jetty-launcher-170-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@62a3b672{HTTP/1.1}{127.0.0.1:41928}
   [junit4]   2> 721574 INFO  (jetty-launcher-170-thread-1) [    ] o.e.j.s.Server Started @724538ms
   [junit4]   2> 721574 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=41928, hostContext=/solr}
   [junit4]   2> 721575 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 721575 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/tempDir-001/'
   [junit4]   2> 721622 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 721644 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 721644 INFO  (zkCallback-172-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d869875 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721645 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 721645 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 721646 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 721680 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/tempDir-001/.
   [junit4]   2> 721680 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 843325190
   [junit4]   2> 721680 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/tempDir-001/]
   [junit4]   2> 721680 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/tempDir-001/lib
   [junit4]   2> 721681 WARN  (jetty-launcher-170-thread-1) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/tempDir-001/lib).
   [junit4]   2> 721690 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 721691 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 721692 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 721692 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 721692 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46808/solr
   [junit4]   2> 721692 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 721692 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 721696 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 721704 INFO  (zkCallback-174-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@369fe650 name:ZooKeeperConnection Watcher:127.0.0.1:46808 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721724 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 721724 INFO  (jetty-launcher-170-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 721734 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 721738 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7bd3566b name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721738 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 721740 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 721750 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 721751 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 721757 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 721765 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 721767 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 721768 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 721769 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 721770 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 721772 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 721772 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41928_solr
   [junit4]   2> 721773 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41928_solr
   [junit4]   2> 721778 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 721779 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 721782 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 721863 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94338797258276867-127.0.0.1:41928_solr-n_0000000000
   [junit4]   2> 721864 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41928_solr
   [junit4]   2> 721864 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 721866 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer Overseer (id=94338797258276867-127.0.0.1:41928_solr-n_0000000000) starting
   [junit4]   2> 721870 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 721879 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 721886 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Process current queue of overseer operations
   [junit4]   2> 721886 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 721887 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 721893 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 721910 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 721910 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 721911 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/tempDir-001/.
   [junit4]   2> 721912 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.c.CoresLocator Found 0 core definitions
   [junit4]   2> 721913 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 721913 INFO  (jetty-launcher-170-thread-1) [n:127.0.0.1:41928_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 721913 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 721915 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 721928 INFO  (zkCallback-176-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7088291e name:ZooKeeperConnection Watcher:127.0.0.1:46808 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721928 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 721929 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 721945 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 721964 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 721975 INFO  (zkCallback-178-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3377b99b name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721975 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 721975 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 721978 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection0/schema.xml
   [junit4]   2> 721982 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection0/solrconfig.xml
   [junit4]   2> 721995 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722000 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722008 INFO  (zkCallback-179-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c1db408 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722008 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722008 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722009 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection1/schema.xml
   [junit4]   2> 722022 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection1/solrconfig.xml
   [junit4]   2> 722026 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722028 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722032 INFO  (zkCallback-180-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b90d454 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722033 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722033 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722033 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection2/schema.xml
   [junit4]   2> 722037 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection2/solrconfig.xml
   [junit4]   2> 722052 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722054 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722056 INFO  (zkCallback-181-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19b1b07 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722056 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722056 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722057 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection3/schema.xml
   [junit4]   2> 722065 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection3/solrconfig.xml
   [junit4]   2> 722104 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722112 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722125 INFO  (zkCallback-182-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72d100fa name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722125 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722125 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722126 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection4/schema.xml
   [junit4]   2> 722129 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection4/solrconfig.xml
   [junit4]   2> 722133 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722133 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722135 INFO  (zkCallback-183-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b9dd924 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722135 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722136 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722136 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection5/schema.xml
   [junit4]   2> 722143 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection5/solrconfig.xml
   [junit4]   2> 722162 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722177 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722189 INFO  (zkCallback-184-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74f36bb5 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722189 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722189 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722190 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection6/schema.xml
   [junit4]   2> 722198 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection6/solrconfig.xml
   [junit4]   2> 722201 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722208 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722216 INFO  (zkCallback-185-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@385fcd26 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722216 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722217 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722217 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection7/schema.xml
   [junit4]   2> 722229 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection7/solrconfig.xml
   [junit4]   2> 722239 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722256 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722264 INFO  (zkCallback-186-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7952176 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722265 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722266 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722266 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection8/schema.xml
   [junit4]   2> 722270 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection8/solrconfig.xml
   [junit4]   2> 722285 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 722287 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 722287 INFO  (zkCallback-187-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@68e01180 name:ZooKeeperConnection Watcher:127.0.0.1:46808/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 722287 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 722288 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 722288 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection9/schema.xml
   [junit4]   2> 722292 INFO  (TEST-ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail-seed#[6F74601963E2F341]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/collection9/solrconfig.xml
   [junit4]   2> 722307 INFO  (qtp439316350-48001) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection2&replicationFactor=1&collection.configName=collection2 
   [junit4]   2> 722308 INFO  (qtp439316350-48002) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection3&replicationFactor=1&collection.configName=collection3 
   [junit4]   2> 722308 INFO  (qtp439316350-48003) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection6&replicationFactor=1&collection.configName=collection6 
   [junit4]   2> 722309 INFO  (qtp439316350-47999) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection1&replicationFactor=1&collection.configName=collection1 
   [junit4]   2> 722310 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
   [junit4]   2> 722310 INFO  (qtp439316350-48000) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection0&replicationFactor=1&collection.configName=collection0 
   [junit4]   2> 722313 INFO  (qtp439316350-48066) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection8&replicationFactor=1&collection.configName=collection8 
   [junit4]   2> 722323 INFO  (qtp439316350-48067) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection5&replicationFactor=1&collection.configName=collection5 
   [junit4]   2> 722324 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2>   "name":"collection2",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection2",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722328 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000001 message:{
   [junit4]   2>   "name":"collection3",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722329 INFO  (qtp439316350-48068) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection9&replicationFactor=1&collection.configName=collection9 
   [junit4]   2> 722329 INFO  (qtp439316350-48069) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection7&replicationFactor=1&collection.configName=collection7 
   [junit4]   2> 722330 WARN  (OverseerThreadFactory-515-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection2",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection2",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722330 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
   [junit4]   2>   "name":"collection6",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection6",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722330 WARN  (OverseerThreadFactory-515-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection3",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722331 INFO  (qtp439316350-48071) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params wt=javabin&action=CREATE&version=2&numShards=1&name=collection4&replicationFactor=1&collection.configName=collection4 
   [junit4]   2> 722331 WARN  (OverseerThreadFactory-515-thread-3-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection6",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection6",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722331 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000003 message:{
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722333 INFO  (OverseerThreadFactory-515-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection2 
   [junit4]   2> 722334 WARN  (OverseerThreadFactory-515-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722334 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000004 message:{
   [junit4]   2>   "name":"collection0",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection0",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722334 INFO  (OverseerThreadFactory-515-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2
   [junit4]   2> 722335 INFO  (OverseerThreadFactory-515-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection3 
   [junit4]   2> 722335 INFO  (OverseerThreadFactory-515-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection3
   [junit4]   2> 722337 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 722338 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"collection2",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection2",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 722339 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection2
   [junit4]   2> 722339 INFO  (OverseerThreadFactory-515-thread-3-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection6 
   [junit4]   2> 722340 INFO  (OverseerThreadFactory-515-thread-3-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection6
   [junit4]   2> 722340 WARN  (OverseerThreadFactory-515-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection0",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection0",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722341 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection2/state.json
   [junit4]   2> 722342 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722343 INFO  (OverseerThreadFactory-515-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection1 
   [junit4]   2> 722343 INFO  (zkCallback-175-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722343 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000010 message:{
   [junit4]   2>   "name":"collection8",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection8",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722344 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000012 message:{
   [junit4]   2>   "name":"collection5",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection5",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722344 INFO  (OverseerThreadFactory-515-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 722345 WARN  (OverseerThreadFactory-515-thread-6-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection8",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection8",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722345 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000014 message:{
   [junit4]   2>   "name":"collection9",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection9",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722345 INFO  (OverseerThreadFactory-515-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection0 
   [junit4]   2> 722345 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000015 message:{
   [junit4]   2>   "name":"collection7",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection7",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722345 WARN  (OverseerThreadFactory-515-thread-7-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection5",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection5",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722346 INFO  (OverseerThreadFactory-515-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection0
   [junit4]   2> 722346 WARN  (OverseerThreadFactory-515-thread-8-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection9",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection9",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722347 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "name":"collection3",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 1
   [junit4]   2> 722347 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection3
   [junit4]   2> 722347 INFO  (OverseerCollectionProcessor-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerProcessor Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000018 message:{
   [junit4]   2>   "name":"collection4",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection4",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722347 WARN  (OverseerThreadFactory-515-thread-10-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection4",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection4",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722348 INFO  (OverseerThreadFactory-515-thread-6-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection8 
   [junit4]   2> 722349 INFO  (OverseerThreadFactory-515-thread-6-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection8
   [junit4]   2> 722349 WARN  (OverseerThreadFactory-515-thread-9-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2>   "name":"collection7",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection7",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 722350 INFO  (zkCallback-175-thread-3-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 722350 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "name":"collection6",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection6",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 1
   [junit4]   2> 722351 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection6
   [junit4]   2> 722351 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection3/state.json
   [junit4]   2> 722353 INFO  (zkCallback-175-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722353 INFO  (zkCallback-175-thread-3-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722353 INFO  (OverseerThreadFactory-515-thread-10-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection4 
   [junit4]   2> 722355 INFO  (OverseerThreadFactory-515-thread-10-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection4
   [junit4]   2> 722359 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 2
   [junit4]   2> 722359 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 722362 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection6/state.json
   [junit4]   2> 722362 INFO  (zkCallback-175-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 722363 INFO  (zkCallback-175-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722363 INFO  (zkCallback-175-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722367 INFO  (OverseerThreadFactory-515-thread-8-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection9 
   [junit4]   2> 722369 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "name":"collection0",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection0",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 3
   [junit4]   2> 722369 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection0
   [junit4]   2> 722369 INFO  (OverseerThreadFactory-515-thread-8-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection9
   [junit4]   2> 722370 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 722370 INFO  (OverseerThreadFactory-515-thread-7-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection5 
   [junit4]   2> 722372 INFO  (OverseerThreadFactory-515-thread-7-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection5
   [junit4]   2> 722395 INFO  (zkCallback-175-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722395 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722432 INFO  (zkCallback-175-thread-6-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 722432 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "name":"collection8",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection8",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 4
   [junit4]   2> 722432 INFO  (OverseerThreadFactory-515-thread-9-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/collection7 
   [junit4]   2> 722433 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection8
   [junit4]   2> 722435 INFO  (OverseerThreadFactory-515-thread-9-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection7
   [junit4]   2> 722435 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection0/state.json
   [junit4]   2> 722437 INFO  (zkCallback-175-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722438 INFO  (OverseerThreadFactory-515-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating SolrCores for new collection collection2, shardNames [shard1] , replicationFactor : 1
   [junit4]   2> 722438 INFO  (OverseerThreadFactory-515-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core collection2_shard1_replica1 as part of shard shard1 of collection collection2 on 127.0.0.1:41928_solr
   [junit4]   2> 722438 INFO  (zkCallback-175-thread-6-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722439 INFO  (OverseerThreadFactory-515-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating SolrCores for new collection collection3, shardNames [shard1] , replicationFactor : 1
   [junit4]   2> 722439 INFO  (OverseerThreadFactory-515-thread-2-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core collection3_shard1_replica1 as part of shard shard1 of collection collection3 on 127.0.0.1:41928_solr
   [junit4]   2> 722449 INFO  (OverseerThreadFactory-515-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating SolrCores for new collection collection1, shardNames [shard1] , replicationFactor : 1
   [junit4]   2> 722449 INFO  (qtp439316350-48073) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CoreAdminHandler core create command wt=javabin&action=CREATE&collection=collection2&shard=shard1&version=2&name=collection2_shard1_replica1&numShards=1&qt=/admin/cores&collection.configName=collection2
   [junit4]   2> 722450 INFO  (qtp439316350-48073) [n:127.0.0.1:41928_solr    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentDeleteAndCreateCollectionTest_6F74601963E2F341-001/tempDir-001/./collection2_shard1_replica1/, loadOnStartup=true, numShards=1, name=collection2_shard1_replica1, configSetProperties=configsetprops.json, schema=schema.xml, transient=false, instanceDir=collection2_shard1_replica1, shard=shard1, dataDir=data/, config=solrconfig.xml, collection=collection2}
   [junit4]   2> 722449 INFO  (OverseerThreadFactory-515-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core collection1_shard1_replica1 as part of shard shard1 of collection collection1 on 127.0.0.1:41928_solr
   [junit4]   2> 722451 INFO  (OverseerThreadFactory-515-thread-3-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating SolrCores for new collection collection6, shardNames [shard1] , replicationFactor : 1
   [junit4]   2> 722451 INFO  (OverseerThreadFactory-515-thread-3-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core collection6_shard1_replica1 as part of shard shard1 of collection collection6 on 127.0.0.1:41928_solr
   [junit4]   2> 722456 INFO  (OverseerThreadFactory-515-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating SolrCores for new collection collection0, shardNames [shard1] , replicationFactor : 1
   [junit4]   2> 722456 INFO  (OverseerThreadFactory-515-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core collection0_shard1_replica1 as part of shard shard1 of collection collection0 on 127.0.0.1:41928_solr
   [junit4]   2> 722459 INFO  (qtp439316350-48073) [n:127.0.0.1:41928_solr c:collection2 s:shard1  x:collection2_shard1_replica1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 722460 INFO  (zkCallback-175-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 722461 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "name":"collection4",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection4",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 5
   [junit4]   2> 722461 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection4
   [junit4]   2> 722462 INFO  (qtp439316350-48073) [n:127.0.0.1:41928_solr c:collection2 s:shard1  x:collection2_shard1_replica1] o.a.s.c.ZkController Registering watch for external collection collection2
   [junit4]   2> 722462 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection8/state.json
   [junit4]   2> 722462 INFO  (qtp439316350-48073) [n:127.0.0.1:41928_solr c:collection2 s:shard1  x:collection2_shard1_replica1] o.a.s.c.c.ZkStateReader addZkWatch collection2
   [junit4]   2> 722463 INFO  (qtp439316350-48073) [n:127.0.0.1:41928_solr c:collection2 s:shard1  x:collection2_shard1_replica1] o.a.s.c.c.ZkStateReader Updating data for collection2 to ver 0 
   [junit4]   2> 722463 INFO  (qtp439316350-48073) [n:127.0.0.1:41928_solr c:collection2 s:shard1  x:collection2_shard1_replica1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 722485 INFO  (zkCallback-175-thread-5-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722486 INFO  (zkCallback-175-thread-6-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722489 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "name":"collection9",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"collection9",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 6
   [junit4]   2> 722489 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection9
   [junit4]   2> 722490 INFO  (OverseerStateUpdate-94338797258276867-127.0.0.1:41928_solr-n_0000000000) [n:127.0.0.1:41928_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection4/state.json
   [junit4]   2> 722492 INFO  (zkCallback-175-thread-4-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722492 INFO  (zkCallback-175-thread-1-processing-n:127.0.0.1:41928_solr) [n:127.0.0.1:41928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 722494 INFO  (qtp439316350-48089) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CoreAdminHandler core create command wt=javabin&action=CREATE&collection=collection3&shard=shard1&version=2&name=collection3_shard1_replica1&numShards=1&qt=/admin/cores&collection.configName=collection3
   [junit4]   2> 722495 INFO  (qtp439316350-48092) [n:127.0.0.1:41928_solr    ] o.a.s.h.a.CoreAdminHandler core create command wt=javabin&action=CREATE&collection=collection0&shard=shard1&version=2&name=collection0_shard1_replica1&numShards=1&qt=/admin/cores&collection.configName=collection0
   [junit4]   2> 722495 INFO  (qtp439316350-48089) [n:127.0.0.1:41928_solr    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, transient=false, collection=collection3, shard=shard1, dataDir=data/, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-c

[...truncated too long message...]

t com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4503472 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[6F74601963E2F341]-worker) [n:127.0.0.1:35475_ c:addReplicaColl s:shard2 r:core_node1 x:addReplicaColl_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 13, 2015 4:59:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_i_dv=Lucene50(blocksize=128), other_tl1=PostingsFormat(name=LuceneFixedGap), _version_=PostingsFormat(name=LuceneFixedGap), intDefault=PostingsFormat(name=LuceneFixedGap), timestamp=Lucene50(blocksize=128), range_facet_l_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_i1=Lucene50(blocksize=128), range_facet_l=Lucene50(blocksize=128), a_t=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), multiDefault=Lucene50(blocksize=128)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=Direct)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=sq_AL, timezone=Asia/Kathmandu
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=91228480,total=526909440
   [junit4]   2> NOTE: All tests run in this JVM: [RAMDirectoryFactoryTest, TestManagedResource, TestFieldCollectionResource, BasicAuthIntegrationTest, TestLRUCache, SpellPossibilityIteratorTest, TestCoreContainer, TestSha256AuthenticationProvider, TestRTGBase, TestFileDictionaryLookup, TestMaxScoreQueryParser, ScriptEngineTest, TestCrossCoreJoin, CurrencyFieldOpenExchangeTest, TestSchemaVersionResource, DistributedMLTComponentTest, FileUtilsTest, TestDistributedStatsComponentCardinality, TestDistributedGrouping, AssignTest, CloudExitableDirectoryReaderTest, TestLRUStatsCache, TestMiniSolrCloudCluster, TestCursorMarkWithoutUniqueKey, SolrXmlInZkTest, TestConfigOverlay, TestLeaderElectionZkExpiry, TestCollapseQParserPlugin, TestCoreDiscovery, CollectionStateFormat2Test, TestFoldingMultitermQuery, TestLMJelinekMercerSimilarityFactory, MergeStrategyTest, BigEndianAscendingWordDeserializerTest, TestJsonRequest, DisMaxRequestHandlerTest, ZkControllerTest, JsonLoaderTest, TestOmitPositions, CleanupOldIndexTest, TestAddFieldRealTimeGet, TestManagedResourceStorage, ZkCLITest, DirectSolrConnectionTest, DistributedFacetPivotSmallTest, TestSearchPerf, TestComponentsName, LukeRequestHandlerTest, SliceStateTest, TestFiltering, PKIAuthenticationIntegrationTest, RecoveryAfterSoftCommitTest, TestPHPSerializedResponseWriter, TestManagedSynonymFilterFactory, SolrCmdDistributorTest, OutOfBoxZkACLAndCredentialsProvidersTest, FieldAnalysisRequestHandlerTest, AnalysisAfterCoreReloadTest, NoCacheHeaderTest, TestHashPartitioner, TestSolrConfigHandler, TestPivotHelperCode, TestExpandComponent, TestUniqueKeyFieldResource, AlternateDirectoryTest, OverseerRolesTest, TestNoOpRegenerator, ZkSolrClientTest, TestJsonFacets, DocumentBuilderTest, DistributedDebugComponentTest, JSONWriterTest, HdfsThreadLeakTest, TestAuthorizationFramework, TestSolrCoreProperties, TestFreeTextSuggestions, DistributedFacetPivotWhiteBoxTest, TestRTimerTree, SolrCloudExampleTest, AnalyticsQueryTest, TestNonDefinedSimilarityFactory, TestWriterPerf, TestConfigSetProperties, DocValuesMultiTest, TestSimpleQParserPlugin, TestFuzzyAnalyzedSuggestions, TestIndexSearcher, UpdateParamsTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, ClusterStateUpdateTest, LeaderElectionTest, TestZkChroot, TestRandomDVFaceting, ShardRoutingCustomTest, AutoCommitTest, TestFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestStressVersions, DistributedTermsComponentTest, TestSolr4Spatial, SpellCheckComponentTest, ConvertedLegacyTest, OverseerCollectionProcessorTest, SoftAutoCommitTest, DocValuesTest, SuggesterTSTTest, SpatialFilterTest, TestCSVLoader, SchemaVersionSpecificBehaviorTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, DefaultValueUpdateProcessorTest, TestRemoteStreaming, DistanceFunctionTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, ReturnFieldsTest, TestCSVResponseWriter, QueryParsingTest, TestPartialUpdateDeduplication, SearchHandlerTest, TestLFUCache, TestSolrDeletionPolicy2, MultiTermTest, TestDocSet, TestElisionMultitermQuery, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, TestCharFilters, TestCodecSupport, TestXIncludeConfig, EchoParamsTest, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, ResourceLoaderTest, TestFastOutputStream, TestFastLRUCache, DateMathParserTest, PreAnalyzedFieldTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, DOMUtilTest, TestSolrJ, ZkNodePropsTest, SystemInfoHandlerTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, AliasIntegrationTest, AsyncMigrateRouteKeyTest, CollectionsAPIAsyncDistributedZkTest, ConnectionManagerTest, CustomCollectionTest, MigrateRouteKeyTest, RemoteQueryErrorTest, SimpleCollectionCreateDeleteTest, TestCloudPivotFacet, TestCryptoKeys, TestRebalanceLeaders, TestRequestStatusCollectionAPI, HdfsChaosMonkeySafeLeaderTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [533/533] on J1 in 1342.39s, 1 test, 1 error <<< FAILURES!

[...truncated 16 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 533 suites, 2111 tests, 4 errors, 2 failures, 25 ignored (14 assumptions)

Total time: 195 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1201514947 bytes
Compression is 0.0%
Took 3 min 19 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 926 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/926/

5 tests failed.
REGRESSION:  org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail

Error Message:
concurrent create and delete collection failed: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55888/solr: Could not fully remove collection: collection4

Stack Trace:
java.lang.AssertionError: concurrent create and delete collection failed: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55888/solr: Could not fully remove collection: collection4
	at __randomizedtesting.SeedInfo.seed([FB030AAA9367F8FF:3A2A5F10E4BC22EE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest.testConcurrentCreateAndDeleteDoesNotFail(ConcurrentDeleteAndCreateCollectionTest.java:72)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=5015, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5015, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44666/sm_x: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([FB030AAA9367F8FF]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1439365030209,generation=2,filelist=[_as.doc, _as.fdt, _as.fdx, _as.fnm, _as.nvd, _as.nvm, _as.si, _as.tim, _as.tip, _at.doc, _at.fdt, _at.fdx, _at.fnm, _at.nvd, _at.nvm, _at.si, _at.tim, _at.tip, _au.doc, _au.fdt, _au.fdx, _au.fnm, _au.nvd, _au.nvm, _au.si, _au.tim, _au.tip, _av.doc, _av.fdt, _av.fdx, _av.fnm, _av.nvd, _av.nvm, _av.si, _av.tim, _av.tip, segments_2]}]> but was:<[{indexVersion=1439365030209,generation=2,filelist=[_as.doc, _as.fdt, _as.fdx, _as.fnm, _as.nvd, _as.nvm, _as.si, _as.tim, _as.tip, _at.doc, _at.fdt, _at.fdx, _at.fnm, _at.nvd, _at.nvm, _at.si, _at.tim, _at.tip, _au.doc, _au.fdt, _au.fdx, _au.fnm, _au.nvd, _au.nvm, _au.si, _au.tim, _au.tip, _av.doc, _av.fdt, _av.fdx, _av.fnm, _av.nvd, _av.nvm, _av.si, _av.tim, _av.tip, segments_2]}, {indexVersion=1439365030209,generation=3,filelist=[_aw.doc, _aw.fdt, _aw.fdx, _aw.fnm, _aw.nvd, _aw.nvm, _aw.si, _aw.tim, _aw.tip, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1439365030209,generation=2,filelist=[_as.doc, _as.fdt, _as.fdx, _as.fnm, _as.nvd, _as.nvm, _as.si, _as.tim, _as.tip, _at.doc, _at.fdt, _at.fdx, _at.fnm, _at.nvd, _at.nvm, _at.si, _at.tim, _at.tip, _au.doc, _au.fdt, _au.fdx, _au.fnm, _au.nvd, _au.nvm, _au.si, _au.tim, _au.tip, _av.doc, _av.fdt, _av.fdx, _av.fnm, _av.nvd, _av.nvm, _av.si, _av.tim, _av.tip, segments_2]}]> but was:<[{indexVersion=1439365030209,generation=2,filelist=[_as.doc, _as.fdt, _as.fdx, _as.fnm, _as.nvd, _as.nvm, _as.si, _as.tim, _as.tip, _at.doc, _at.fdt, _at.fdx, _at.fnm, _at.nvd, _at.nvm, _at.si, _at.tim, _at.tip, _au.doc, _au.fdt, _au.fdx, _au.fnm, _au.nvd, _au.nvm, _au.si, _au.tim, _au.tip, _av.doc, _av.fdt, _av.fdx, _av.fnm, _av.nvd, _av.nvm, _av.si, _av.tim, _av.tip, segments_2]}, {indexVersion=1439365030209,generation=3,filelist=[_aw.doc, _aw.fdt, _aw.fdx, _aw.fnm, _aw.nvd, _aw.nvm, _aw.si, _aw.tim, _aw.tip, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([FB030AAA9367F8FF:DED4119AE32FF6FC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:45934: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45934: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:301)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:418)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:168)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=3093, name=WRITER7, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3093, name=WRITER7, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
	at __randomizedtesting.SeedInfo.seed([FB030AAA9367F8FF:C682AE0BC72C1BBD]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([FB030AAA9367F8FF]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1662)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1519)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:778)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1089)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1631)
	... 21 more




Build Log:
[...truncated 11041 lines...]
   [junit4] Suite: org.apache.solr.search.TestReloadDeadlock
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001
   [junit4]   2> 972393 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 972394 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 972394 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 972394 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 972395 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 972395 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 972442 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 972469 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 972505 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 972506 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 972512 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 972717 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 972719 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 972729 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 972732 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 972740 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 972740 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 972740 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 972784 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 681787977
   [junit4]   2> 972784 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 972784 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
   [junit4]   2> 972784 WARN  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib).
   [junit4]   2> 972792 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 972794 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 972795 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 972795 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 972795 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 972795 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 972796 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=collection1, config=solrconfig-tlog.xml, schema=schema15.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, transient=false, name=collection1, configSetProperties=configsetprops.json, shard=shard1, dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001, collection=collection1, loadOnStartup=true}
   [junit4]   2> 972797 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 972804 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 972804 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 972804 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 972823 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 972836 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 972865 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 972866 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 972872 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 973154 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 973157 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 973159 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 973163 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 973175 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 973175 INFO  (coreLoadExecutor-1028-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 973176 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 973176 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 973176 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7cda2a84
   [junit4]   2> 973177 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001
   [junit4]   2> 973178 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001/index/
   [junit4]   2> 973178 WARN  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 973178 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001/index
   [junit4]   2> 973178 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7a767a62
   [junit4]   2> 973179 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_1,generation=1}
   [junit4]   2> 973179 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 973186 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 973186 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 973187 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 973187 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 973187 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 973187 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 973188 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 973188 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 973188 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 973193 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 973195 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 973197 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001/
   [junit4]   2> 973198 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 973202 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 973216 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/update/csv,/admin/file,/update,/admin/plugins,/update/json,/admin/mbeans,/admin/luke,/replication,/admin/segments,/admin/system,/config,/schema,/admin/properties,/get,/admin/logging,/update/json/docs,/admin/ping,/admin/threads
   [junit4]   2> 973222 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 973223 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 973223 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 973224 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 973224 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 973225 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=2089740733, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 973226 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_1,generation=1}
   [junit4]   2> 973226 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 973226 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@13573da3[collection1] main
   [junit4]   2> 973227 WARN  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 973227 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 973227 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 973228 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 973228 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 973229 INFO  (searcherExecutor-1029-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@13573da3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 973229 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 973229 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 973230 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 973230 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1509281733245140992
   [junit4]   2> 973232 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 3 ms to seed version buckets with highest version 1509281733245140992
   [junit4]   2> 973233 INFO  (coreLoadExecutor-1028-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 973234 INFO  (SUITE-TestReloadDeadlock-seed#[FB030AAA9367F8FF]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 973239 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testReloadDeadlock
   [junit4]   2> 973240 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 973240 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@49696c88[collection1] realtime
   [junit4]   2> 973240 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=*:* (-9223372036854775807)} 0 1
   [junit4]   2> 973241 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 973242 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_2,generation=2}
   [junit4]   2> 973243 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 973243 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3d4323e1[collection1] main
   [junit4]   2> 973243 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 973244 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3d4323e1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 973245 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 4
   [junit4]   2> 973263 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[30 (1)]} 0 13
   [junit4]   2> 973292 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2f2e02c8[collection1] realtime
   [junit4]   2> 973293 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-2&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:42 (-2)} 0 35
   [junit4]   2> 973293 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 973295 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[24 (3)]} 0 37
   [junit4]   2> 973298 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_3,generation=3}
   [junit4]   2> 973299 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 973299 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[29 (4)]} 0 34
   [junit4]   2> 973305 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@31c8c618[collection1] main
   [junit4]   2> 973305 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 973306 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@31c8c618[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):C1) Uninverting(_1(5.4.0):C1) Uninverting(_2(5.4.0):C1)))}
   [junit4]   2> 973307 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 42
   [junit4]   2> 973313 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6ebd7211[collection1] realtime
   [junit4]   2> 973314 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-5&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:29 (-5)} 0 48
   [junit4]   2> 973315 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-6&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:29 (-6)} 0 26
   [junit4]   2> 973316 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[38 (7)]} 0 26
   [junit4]   2> 973321 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@525294d9[collection1] realtime
   [junit4]   2> 973322 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-9&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:31 (-9)} 0 30
   [junit4]   2> 973323 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-8&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-8)} 0 33
   [junit4]   2> 973324 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[42 (10)]} 0 31
   [junit4]   2> 973325 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[42 (11)]} 0 29
   [junit4]   2> 973325 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[42 (13)]} 0 14
   [junit4]   2> 973325 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[21 (12)]} 0 23
   [junit4]   2> 973325 INFO  (WRITER4) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 973331 INFO  (WRITER4) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_3,generation=3}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_4,generation=4}
   [junit4]   2> 973331 INFO  (WRITER4) [    ] o.a.s.c.SolrCore newest commit generation = 4
   [junit4]   2> 973332 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@31882856[collection1] main
   [junit4]   2> 973332 INFO  (WRITER4) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 973333 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@31882856[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):C1) Uninverting(_1(5.4.0):C1) Uninverting(_3(5.4.0):C1) Uninverting(_4(5.4.0):C4/2:delGen=1)))}
   [junit4]   2> 973334 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 19
   [junit4]   2> 973336 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-14&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:29 (-14)} 0 20
   [junit4]   2> 973340 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-15&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-15)} 0 24
   [junit4]   2> 973342 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-16&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:44 (-16)} 0 20
   [junit4]   2> 973343 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (17)]} 0 18
   [junit4]   2> 973347 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@78610b23[collection1] realtime
   [junit4]   2> 973347 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-18&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-18)} 0 23
   [junit4]   2> 973348 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (19)]} 0 23
   [junit4]   2> 973353 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@79ad29c3[collection1] realtime
   [junit4]   2> 973353 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-20&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:42 (-20)} 0 28
   [junit4]   2> 973361 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[20 (21)]} 0 35
   [junit4]   2> 973361 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 973362 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[29 (22)]} 0 23
   [junit4]   2> 973362 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (24)]} 0 20
   [junit4]   2> 973362 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[29 (23)]} 0 24
   [junit4]   2> 973362 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 973362 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 973362 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (28)]} 0 14
   [junit4]   2> 973362 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 973363 INFO  (WRITER5) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 973363 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (29)]} 0 9
   [junit4]   2> 973366 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[20 (30)]} 0 4
   [junit4]   2> 973369 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[29 (31)]} 0 6
   [junit4]   2> 973376 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (26)]} 0 33
   [junit4]   2> 973376 INFO  (WRITER5) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_4,generation=4}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_5,generation=5}
   [junit4]   2> 973377 INFO  (WRITER5) [    ] o.a.s.c.SolrCore newest commit generation = 5
   [junit4]   2> 973377 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (27)]} 0 28
   [junit4]   2> 973377 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[1 (25)]} 0 34
   [junit4]   2> 973383 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@e78edbf[collection1] main
   [junit4]   2> 973383 INFO  (WRITER5) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 973385 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@e78edbf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):C1) Uninverting(_1(5.4.0):C1) Uninverting(_3(5.4.0):C1) Uninverting(_4(5.4.0):C4/3:delGen=2) Uninverting(_9(5.4.0):C2) Uninverting(_a(5.4.0):C3)))}
   [junit4]   2> 973392 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 30
   [junit4]   2> 973395 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@68296868[collection1] realtime
   [junit4]   2> 973396 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (33)]} 0 33
   [junit4]   2> 973397 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (34)]} 0 32
   [junit4]   2> 973403 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@64217ca[collection1] realtime
   [junit4]   2> 973405 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-32&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-32)} 0 42
   [junit4]   2> 973405 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (36)]} 0 36
   [junit4]   2> 973406 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-35&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:20 (-35)} 0 39
   [junit4]   2> 973406 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (37)]} 0 29
   [junit4]   2> 973406 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[38 (39)]} 0 28
   [junit4]   2> 973407 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (38)]} 0 29
   [junit4]   2> 973411 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 973421 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@918bb06[collection1] realtime
   [junit4]   2> 973423 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (42)]} 0 25
   [junit4]   2> 973429 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@388e42a3[collection1] realtime
   [junit4]   2> 973429 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-40&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:24 (-40)} 0 32
   [junit4]   2> 973433 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2ac01d9b[collection1] realtime
   [junit4]   2> 973433 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-44&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-44)} 0 27
   [junit4]   2> 973436 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-45&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:20 (-45)} 0 29
   [junit4]   2> 973438 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (47)]} 0 31
   [junit4]   2> 973438 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (46)]} 0 32
   [junit4]   2> 973439 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-41&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:24 (-41)} 0 41
   [junit4]   2> 973439 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[24 (48)]} 0 32
   [junit4]   2> 973441 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 973451 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7d1ffa33[collection1] realtime
   [junit4]   2> 973451 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-43&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-43)} 0 44
   [junit4]   2> 973455 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (49)]} 0 31
   [junit4]   2> 973455 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (53)]} 0 17
   [junit4]   2> 973456 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[36 (50)]} 0 26
   [junit4]   2> 973457 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (52)]} 0 21
   [junit4]   2> 973458 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[23 (51)]} 0 24
   [junit4]   2> 973471 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2147df18[collection1] realtime
   [junit4]   2> 973472 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-54&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:35 (-54)} 0 33
   [junit4]   2> 973473 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[24 (55)]} 0 33
   [junit4]   2> 973473 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[24 (56)]} 0 33
   [junit4]   2> 973478 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@b9e4ba4[collection1] realtime
   [junit4]   2> 973481 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-59&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:44 (-59)} 0 24
   [junit4]   2> 973482 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-58&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:44 (-58)} 0 26
   [junit4]   2> 973482 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-57&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:40 (-57)} 0 30
   [junit4]   2> 973483 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (60)]} 0 27
   [junit4]   2> 973484 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[23 (62)]} 0 25
   [junit4]   2> 973484 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (61)]} 0 26
   [junit4]   2> 973484 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (63)]} 0 12
   [junit4]   2> 973484 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[24 (64)]} 0 11
   [junit4]   2> 973485 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[24 (65)]} 0 11
   [junit4]   2> 973486 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[44 (66)]} 0 4
   [junit4]   2> 973486 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[44 (67)]} 0 3
   [junit4]   2> 973486 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 973496 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[22 (72)]} 0 10
   [junit4]   2> 973497 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (73)]} 0 11
   [junit4]   2> 973498 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (69)]} 0 12
   [junit4]   2> 973498 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (70)]} 0 14
   [junit4]   2> 973499 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (71)]} 0 14
   [junit4]   2> 973503 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (68)]} 0 20
   [junit4]   2> 973504 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 973504 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 973505 INFO  (WRITER3) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_5,generation=5}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_6,generation=6}
   [junit4]   2> 973505 INFO  (WRITER3) [    ] o.a.s.c.SolrCore newest commit generation = 6
   [junit4]   2> 973515 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2f862e97[collection1] main
   [junit4]   2> 973515 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 973516 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 973516 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2f862e97[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_h(5.4.0):c12/4:delGen=1) Uninverting(_i(5.4.0):C1) Uninverting(_m(5.4.0):C1) Uninverting(_o(5.4.0):C1) Uninverting(_r(5.4.0):C7/2:delGen=1) Uninverting(_s(5.4.0):C2/1:delGen=1) Uninverting(_t(5.4.0):C2)))}
   [junit4]   2> 973518 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 33
   [junit4]   2> 973521 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@28d0b119[collection1] realtime
   [junit4]   2> 973521 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-74&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-74)} 0 35
   [junit4]   2> 973522 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (75)]} 0 35
   [junit4]   2> 973522 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[22 (76)]} 0 26
   [junit4]   2> 973530 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2edc7f63[collection1] realtime
   [junit4]   2> 973532 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (78)]} 0 33
   [junit4]   2> 973537 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@cdbedf[collection1] realtime
   [junit4]   2> 973538 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-79&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:40 (-79)} 0 39
   [junit4]   2> 973538 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-77&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:2 (-77)} 0 40
   [junit4]   2> 973539 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[30 (81)]} 0 35
   [junit4]   2> 973540 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[38 (84)]} 0 17
   [junit4]   2> 973540 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[38 (85)]} 0 17
   [junit4]   2> 973540 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (80)]} 0 41
   [junit4]   2> 973540 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (83)]} 0 19
   [junit4]   2> 973541 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (82)]} 0 20
   [junit4]   2> 973560 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4563011c[collection1] realtime
   [junit4]   2> 973561 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-86&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:14 (-86)} 0 29
   [junit4]   2> 973563 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-87&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:39 (-87)} 0 24
   [junit4]   2> 973565 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[39 (88)]} 0 27
   [junit4]   2> 973566 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[37 (90)]} 0 25
   [junit4]   2> 973565 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[30 (89)]} 0 25
   [junit4]   2> 973576 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@704651e7[collection1] realtime
   [junit4]   2> 973576 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-91&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-91)} 0 36
   [junit4]   2> 973576 INFO  (WRITER8) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 973578 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[32 (92)]} 0 37
   [junit4]   2> 973578 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (94)]} 0 16
   [junit4]   2> 973578 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (95)]} 0 14
   [junit4]   2> 973583 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[26 (93)]} 0 26
   [junit4]   2> 973633 INFO  (WRITER8) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_6,generation=6}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_7,generation=7}
   [junit4]   2> 973633 INFO  (WRITER8) [    ] o.a.s.c.SolrCore newest commit generation = 7
   [junit4]   2> 973649 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@48c49ca2[collection1] main
   [junit4]   2> 973650 INFO  (WRITER8) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 973651 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@48c49ca2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_h(5.4.0):c12/6:delGen=2) Uninverting(_m(5.4.0):C1) Uninverting(_o(5.4.0):C1) Uninverting(_r(5.4.0):C7/4:delGen=2) Uninverting(_u(5.4.0):C1) Uninverting(_v(5.4.0):C1) Uninverting(_x(5.4.0):C3/2:delGen=2) Uninverting(_z(5.4.0):C2) Uninverting(_10(5.4.0):C1) Uninverting(_12(5.4.0):C2) Uninverting(_11(5.4.0):C1)))}
   [junit4]   2> 973655 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7725f2e2[collection1] realtime
   [junit4]   2> 973655 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-97&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:18 (-97)} 0 89
   [junit4]   2> 973655 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 114
   [junit4]   2> 973660 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@a46c229[collection1] realtime
   [junit4]   2> 973661 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-98&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:32 (-98)} 0 94
   [junit4]   2> 973662 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@28376e9e[collection1] realtime
   [junit4]   2> 973662 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-96&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:28 (-96)} 0 96
   [junit4]   2> 973663 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (100)]} 0 84
   [junit4]   2> 973666 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@660264de[collection1] realtime
   [junit4]   2> 973666 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-101&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:40 (-101)} 0 88
   [junit4]   2> 973667 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-99&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:32 (-99)} 0 89
   [junit4]   2> 973668 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[40 (102)]} 0 89
   [junit4]   2> 973669 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[26 (103)]} 0 83
   [junit4]   2> 973675 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@35a67ba0[collection1] realtime
   [junit4]   2> 973675 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-104&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:26 (-104)} 0 19
   [junit4]   2> 973677 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (109)]} 0 10
   [junit4]   2> 973677 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[32 (110)]} 0 9
   [junit4]   2> 973678 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[20 (111)]} 0 8
   [junit4]   2> 973678 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[26 (105)]} 0 19
   [junit4]   2> 973678 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[36 (108)]} 0 14
   [junit4]   2> 973678 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[36 (107)]} 0 15
   [junit4]   2> 973679 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[36 (106)]} 0 17
   [junit4]   2> 973743 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@665be334[collection1] realtime
   [junit4]   2> 973745 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[26 (113)]} 0 69
   [junit4]   2> 973745 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[34 (114)]} 0 67
   [junit4]   2> 973745 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[34 (115)]} 0 67
   [junit4]   2> 973750 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6fbf1740[collection1] realtime
   [junit4]   2> 973752 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 973753 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[30 (117)]} 0 74
   [junit4]   2> 973761 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (120)]} 0 81
   [junit4]   2> 973761 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-112&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:26 (-112)} 0 92
   [junit4]   2> 973761 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-116&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-116)} 0 83
   [junit4]   2> 973762 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (118)]} 0 82
   [junit4]   2> 973762 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (119)]} 0 83
   [junit4]   2> 973776 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 973777 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 973778 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 973781 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 973845 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_7,generation=7}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@785cf6bd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f4b3007; maxCacheMB=0.1611328125 maxMergeSizeMB=0.0078125)),segFN=segments_8,generation=8}
   [junit4]   2> 973845 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 8
   [junit4]   2> 973853 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 973853 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 973853 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001/]
   [junit4]   2> 973854 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7cda2a84
   [junit4]   2> 973855 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001/index/
   [junit4]   2> 973855 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_FB030AAA9367F8FF-001/init-core-data-001/
   [junit4]   2> 973856 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@fc3d7fd[collection1] main
   [junit4]   2> 973858 INFO  (searcherExecutor-1029-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@fc3d7fd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_13(5.4.0):c19/5:delGen=1) Uninverting(_14(5.4.0):C1) Uninverting(_15(5.4.0):C1) Uninverting(_1a(5.4.0):C3/1:delGen=1) Uninverting(_19(5.4.0):C1) Uninverting(_18(5.4.0):C1) Uninverting(_1b(5.4.0):C3/1:delGen=1) Uninverting(_1c(5.4.0):C1) Uninverting(_1e(5.4.0):C1) Uninverting(_1f(5.4.0):C1)))}
   [junit4]   2> 973859 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 973864 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 119
   [junit4]   2> 973865 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 973865 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 973865 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 973865 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 973865 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 973866 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 973866 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4e8c6428[collection1] realtime
   [junit4]   2> 973866 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-121&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:26 (-121)} 0 120
   [junit4]   2> 973866 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 973866 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 973866 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 973867 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@79ef634[collection1] realtime
   [junit4]   2> 973867 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-122&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:34 (-122)} 0 121
   [junit4]   2> 973868 INFO  (WRITER8) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@ba8e12c[collection1] realtime
   [junit4]   2> 973868 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-123&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:30 (-123)} 0 115
   [junit4]   2> 973869 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-124&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:31 (-124)} 0 108
   [junit4]   2> 973869 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 973871 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 973871 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (125)]} 0 109
   [junit4]   2> 973872 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[41 (133)]} 0 1
   [junit4]   2> 973872 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[FB030AAA9367F8FF]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 973873 INFO  (WRITER8) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] we

[...truncated too long message...]

Thread[id=11658, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55876, http://127.0.0.1:45934, http://127.0.0.1:57578, http://127.0.0.1:48813, http://127.0.0.1:57704]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([FB030AAA9367F8FF]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:897)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55876, http://127.0.0.1:45934, http://127.0.0.1:57578, http://127.0.0.1:48813, http://127.0.0.1:57704]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57578
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:570)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
   [junit4]    > 	... 9 more
   [junit4]   2> 3152916 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[FB030AAA9367F8FF]-worker) [n:127.0.0.1:48813_ c:awholynewstresscollection_collection3_0 s:shard7 r:core_node14 x:awholynewstresscollection_collection3_0_shard7_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_FB030AAA9367F8FF-001
   [junit4]   2> Aug 12, 2015 3:43:25 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_i_dv=PostingsFormat(name=Direct), intDefault=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=Direct), range_facet_l_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), other_tl1=BlockTreeOrds(blocksize=128), _version_=BlockTreeOrds(blocksize=128), a_t=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=Direct), text=PostingsFormat(name=Memory doPackFST= false), a_i1=PostingsFormat(name=Direct)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=en_IE, timezone=SystemV/EST5EDT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=93700216,total=500170752
   [junit4]   2> NOTE: All tests run in this JVM: [TestFastOutputStream, CoreAdminCreateDiscoverTest, TestQueryUtils, TestSmileRequest, DistributedQueryElevationComponentTest, SecurityConfHandlerTest, SampleTest, TestSchemaManager, TestSimpleTrackingShardHandler, TermVectorComponentDistributedTest, LoggingHandlerTest, TestLuceneMatchVersion, HttpPartitionTest, XmlUpdateRequestHandlerTest, MultiThreadedOCPTest, BJQParserTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestJettySolrRunner, XsltUpdateRequestHandlerTest, TestIntervalFaceting, TestBulkSchemaConcurrent, BasicDistributedZk2Test, TestDynamicFieldResource, AsyncCallRequestStatusResponseTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, AutoCommitTest, TestRecovery, TestDocSet, OpenExchangeRatesOrgProviderTest, CachingDirectoryFactoryTest, TestInfoStreamLogging, ChangedSchemaMergeTest, SortByFunctionTest, CursorMarkTest, TestXIncludeConfig, BufferStoreTest, DocExpirationUpdateProcessorFactoryTest, FileBasedSpellCheckerTest, TestGroupingSearch, BlockCacheTest, HdfsDirectoryTest, TestSolrQueryParserDefaultOperatorResource, PrimUtilsTest, SpatialRPTFieldTypeTest, TestSystemIdResolver, BigEndianAscendingWordSerializerTest, ResponseHeaderTest, DirectSolrSpellCheckerTest, TestRebalanceLeaders, TestSort, NumericFieldsTest, TestSolrQueryParserResource, TestReload, SpellCheckCollatorTest, TestDistribIDF, HdfsCollectionsAPIDistributedZkTest, HdfsSyncSliceTest, TestDocBasedVersionConstraints, QueryResultKeyTest, TestCryptoKeys, TestRemoveLastDynamicCopyField, SoftAutoCommitTest, TestSolrJ, SolrCoreCheckLockOnStartupTest, HardAutoCommitTest, QueryParsingTest, TestRequestStatusCollectionAPI, DistributedExpandComponentTest, SynonymTokenizerTest, TestRandomRequestDistribution, PathHierarchyTokenizerFactoryTest, TestStressRecovery, SolrIndexSplitterTest, OutputWriterTest, TestPartialUpdateDeduplication, FullSolrCloudDistribCmdsTest, MoreLikeThisHandlerTest, DeleteShardTest, DeleteReplicaTest, TestConfigSets, CollectionReloadTest, SolrInfoMBeanTest, RankQueryTest, TestStandardQParsers, BadIndexSchemaTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [406/533] on J2 in 1318.60s, 1 test, 1 error <<< FAILURES!

[...truncated 418 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:776: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:713: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 533 suites, 2111 tests, 3 errors, 2 failures, 26 ignored (14 assumptions)

Total time: 185 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1228028293 bytes
Compression is 0.0%
Took 3 min 11 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 925 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/925/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=8286, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8286, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37696/m: Could not find collection : awholynewstresscollection_collection2_1
	at __randomizedtesting.SeedInfo.seed([43D7C7E8846AE831]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=5371, name=WRITER0, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5371, name=WRITER0, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
	at __randomizedtesting.SeedInfo.seed([43D7C7E8846AE831:7E566349D0210B73]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([43D7C7E8846AE831]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1662)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1519)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:778)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1089)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1631)
	... 21 more




Build Log:
[...truncated 10719 lines...]
   [junit4] Suite: org.apache.solr.search.TestReloadDeadlock
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001
   [junit4]   2> 529176 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 529176 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 529177 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 529177 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 529177 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 529178 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 529214 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 529231 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 529259 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 529260 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 529266 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 529504 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 529506 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 529507 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 529510 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 529521 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 529521 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 529521 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 529545 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 1367684093
   [junit4]   2> 529546 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 529546 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
   [junit4]   2> 529546 WARN  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib).
   [junit4]   2> 529556 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 529558 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 529559 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 529559 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 529559 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 529559 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 529560 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, shard=shard1, transient=false, collection=collection1, schema=schema15.xml, name=collection1, config=solrconfig-tlog.xml, instanceDir=collection1}
   [junit4]   2> 529561 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 529562 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 529563 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 529563 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 529612 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 529635 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 529664 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 529664 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 529673 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 529919 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 529921 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 529922 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 529924 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 529933 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 529934 INFO  (coreLoadExecutor-2039-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 529934 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 529934 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 529934 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4be82e3a
   [junit4]   2> 529935 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001
   [junit4]   2> 529935 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001/index/
   [junit4]   2> 529935 WARN  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 529935 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001/index
   [junit4]   2> 529936 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 529936 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_1,generation=1}
   [junit4]   2> 529936 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 529941 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 529941 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 529941 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 529942 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 529942 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 529942 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 529942 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 529942 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 529942 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 529945 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 529947 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 529948 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 529949 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 529949 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001/
   [junit4]   2> 529953 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/admin/system,/update/csv,/admin/properties,standard,/admin/segments,/replication,/admin/file,/admin/plugins,/update/json,/admin/mbeans,/schema,/get,/admin/ping,/admin/threads,/config,/update/json/docs,/admin/luke,/admin/logging
   [junit4]   2> 529955 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 529955 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 529955 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 529956 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 529956 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 529956 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=53.986328125, floorSegmentMB=1.8056640625, forceMergeDeletesPctAllowed=18.352471163119944, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.82753605065073
   [junit4]   2> 529957 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_1,generation=1}
   [junit4]   2> 529957 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 529957 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6abb9692[collection1] main
   [junit4]   2> 529958 WARN  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 529958 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 529958 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 529958 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 529958 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 529958 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 529959 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 529959 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 529959 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1509191054468841472
   [junit4]   2> 529961 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 3 ms to seed version buckets with highest version 1509191054468841472
   [junit4]   2> 529961 INFO  (coreLoadExecutor-2039-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 529962 INFO  (searcherExecutor-2040-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6abb9692[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 529962 INFO  (SUITE-TestReloadDeadlock-seed#[43D7C7E8846AE831]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 529966 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testReloadDeadlock
   [junit4]   2> 529967 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 529968 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4b27539[collection1] realtime
   [junit4]   2> 529968 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=*:* (-9223372036854775807)} 0 1
   [junit4]   2> 529968 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 529969 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_2,generation=2}
   [junit4]   2> 529969 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 529970 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1f9b6891[collection1] main
   [junit4]   2> 529970 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 529970 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1f9b6891[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 529971 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 2
   [junit4]   2> 529989 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-1&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-1)} 0 13
   [junit4]   2> 529990 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-3&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-3)} 0 14
   [junit4]   2> 529991 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-4&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-4)} 0 14
   [junit4]   2> 529991 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-2&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-2)} 0 15
   [junit4]   2> 530013 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@16a8966[collection1] realtime
   [junit4]   2> 530013 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-5&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-5)} 0 35
   [junit4]   2> 530014 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (7)]} 0 36
   [junit4]   2> 530014 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (6)]} 0 29
   [junit4]   2> 530015 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (8)]} 0 29
   [junit4]   2> 530015 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (10)]} 0 23
   [junit4]   2> 530015 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (12)]} 0 23
   [junit4]   2> 530015 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (11)]} 0 23
   [junit4]   2> 530015 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (9)]} 0 25
   [junit4]   2> 530016 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (13)]} 0 1
   [junit4]   2> 530025 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (14)]} 0 11
   [junit4]   2> 530026 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (15)]} 0 11
   [junit4]   2> 530038 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6035e75d[collection1] realtime
   [junit4]   2> 530039 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-17&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-17)} 0 23
   [junit4]   2> 530039 INFO  (WRITER7) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=16,id=13} DBQs=[DBQ{version=17,q=id:0}]
   [junit4]   2> 530048 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7aac62fa[collection1] realtime
   [junit4]   2> 530048 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (16)]} 0 33
   [junit4]   2> 530048 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (18)]} 0 32
   [junit4]   2> 530049 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (20)]} 0 33
   [junit4]   2> 530049 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (19)]} 0 33
   [junit4]   2> 530055 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@673e560a[collection1] realtime
   [junit4]   2> 530056 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-21&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-21)} 0 39
   [junit4]   2> 530057 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (22)]} 0 31
   [junit4]   2> 530057 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (24)]} 0 18
   [junit4]   2> 530058 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (23)]} 0 32
   [junit4]   2> 530067 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@34d66a18[collection1] realtime
   [junit4]   2> 530068 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (26)]} 0 18
   [junit4]   2> 530075 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@352bea02[collection1] realtime
   [junit4]   2> 530075 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-27&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-27)} 0 25
   [junit4]   2> 530075 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-25&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-25)} 0 26
   [junit4]   2> 530076 INFO  (WRITER7) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 530076 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 530077 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 530077 INFO  (WRITER7) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_3,generation=3}
   [junit4]   2> 530077 INFO  (WRITER7) [    ] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 530077 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 530078 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 530078 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@57eeec82[collection1] main
   [junit4]   2> 530082 INFO  (WRITER7) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 530083 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@57eeec82[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(5.4.0):c4/2:delGen=2) Uninverting(_3(5.4.0):c1) Uninverting(_6(5.4.0):c1) Uninverting(_8(5.4.0):c1) Uninverting(_9(5.4.0):c1)))}
   [junit4]   2> 530083 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 33
   [junit4]   2> 530088 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2ac8100f[collection1] realtime
   [junit4]   2> 530088 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-29&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-29)} 0 31
   [junit4]   2> 530089 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (30)]} 0 31
   [junit4]   2> 530093 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2f01c641[collection1] realtime
   [junit4]   2> 530094 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-28&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-28)} 0 36
   [junit4]   2> 530095 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (32)]} 0 26
   [junit4]   2> 530095 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (31)]} 0 36
   [junit4]   2> 530102 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2573e21[collection1] realtime
   [junit4]   2> 530102 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-33&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:15 (-33)} 0 26
   [junit4]   2> 530103 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-34&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-34)} 0 26
   [junit4]   2> 530103 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-35&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-35)} 0 15
   [junit4]   2> 530104 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-36&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:2 (-36)} 0 15
   [junit4]   2> 530105 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-37&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-37)} 0 15
   [junit4]   2> 530105 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (39)]} 0 10
   [junit4]   2> 530106 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (40)]} 0 10
   [junit4]   2> 530109 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3b9a35ac[collection1] realtime
   [junit4]   2> 530109 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-38&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-38)} 0 13
   [junit4]   2> 530110 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (41)]} 0 8
   [junit4]   2> 530113 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@25f2caf9[collection1] realtime
   [junit4]   2> 530114 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-42&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-42)} 0 10
   [junit4]   2> 530114 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@40cbfc92[collection1] realtime
   [junit4]   2> 530115 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-43&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-43)} 0 11
   [junit4]   2> 530116 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (44)]} 0 11
   [junit4]   2> 530116 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (45)]} 0 10
   [junit4]   2> 530116 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (46)]} 0 10
   [junit4]   2> 530121 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6f7eec50[collection1] realtime
   [junit4]   2> 530121 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 530121 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-47&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-47)} 0 15
   [junit4]   2> 530123 INFO  (WRITER3) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_3,generation=3}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_4,generation=4}
   [junit4]   2> 530123 INFO  (WRITER3) [    ] o.a.s.c.SolrCore newest commit generation = 4
   [junit4]   2> 530123 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (49)]} 0 8
   [junit4]   2> 530123 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (48)]} 0 12
   [junit4]   2> 530128 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 530131 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@738fa6d0[collection1] main
   [junit4]   2> 530131 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 530132 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@738fa6d0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(5.4.0):c4/2:delGen=2) Uninverting(_3(5.4.0):c1) Uninverting(_d(5.4.0):c2/1:delGen=1) Uninverting(_f(5.4.0):c2) Uninverting(_i(5.4.0):c1) Uninverting(_h(5.4.0):c1)))}
   [junit4]   2> 530132 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 22
   [junit4]   2> 530133 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3a87efd0[collection1] realtime
   [junit4]   2> 530134 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-50&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-50)} 0 18
   [junit4]   2> 530134 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-51&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-51)} 0 18
   [junit4]   2> 530135 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[15 (53)]} 0 18
   [junit4]   2> 530135 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (52)]} 0 18
   [junit4]   2> 530140 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7499614b[collection1] realtime
   [junit4]   2> 530140 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-54&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:3 (-54)} 0 18
   [junit4]   2> 530141 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@124351bb[collection1] realtime
   [junit4]   2> 530141 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-55&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-55)} 0 17
   [junit4]   2> 530141 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (56)]} 0 18
   [junit4]   2> 530142 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (57)]} 0 7
   [junit4]   2> 530146 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@14e5ae2[collection1] realtime
   [junit4]   2> 530147 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (60)]} 0 12
   [junit4]   2> 530148 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (59)]} 0 12
   [junit4]   2> 530149 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-58&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:10 (-58)} 0 14
   [junit4]   2> 530149 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (61)]} 0 12
   [junit4]   2> 530150 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (62)]} 0 9
   [junit4]   2> 530150 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (63)]} 0 8
   [junit4]   2> 530150 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (64)]} 0 8
   [junit4]   2> 530151 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (65)]} 0 8
   [junit4]   2> 530151 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 530156 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@66c077a1[collection1] realtime
   [junit4]   2> 530157 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-66&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:2 (-66)} 0 8
   [junit4]   2> 530157 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (67)]} 0 8
   [junit4]   2> 530164 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5ae9bc2f[collection1] realtime
   [junit4]   2> 530164 INFO  (WRITER6) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=68,id=8} DBQs=[DBQ{version=69,q=id:1}]
   [junit4]   2> 530165 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-69&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-69)} 0 15
   [junit4]   2> 530165 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (72)]} 0 14
   [junit4]   2> 530172 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@cbe217c[collection1] realtime
   [junit4]   2> 530172 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (73)]} 0 21
   [junit4]   2> 530173 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (68)]} 0 23
   [junit4]   2> 530173 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-71&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:12 (-71)} 0 22
   [junit4]   2> 530174 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (70)]} 0 24
   [junit4]   2> 530175 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[2 (74)]} 0 18
   [junit4]   2> 530175 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (76)]} 0 9
   [junit4]   2> 530176 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (75)]} 0 18
   [junit4]   2> 530208 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-tlog.xml
   [junit4]   2> 530209 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
   [junit4]   2> 530216 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 530240 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3bae6f77[collection1] realtime
   [junit4]   2> 530240 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-77&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-77)} 0 73
   [junit4]   2> 530242 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7230d16a[collection1] realtime
   [junit4]   2> 530242 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-78&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-78)} 0 69
   [junit4]   2> 530243 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-79&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-79)} 0 70
   [junit4]   2> 530244 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 530245 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (80)]} 0 70
   [junit4]   2> 530245 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (81)]} 0 69
   [junit4]   2> 530246 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (83)]} 0 69
   [junit4]   2> 530248 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (82)]} 0 72
   [junit4]   2> 530249 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_4,generation=4}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_5,generation=5}
   [junit4]   2> 530250 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 5
   [junit4]   2> 530313 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@67cfee89[collection1] main
   [junit4]   2> 530314 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@67cfee89[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_r(5.4.0):c9/3:delGen=1) Uninverting(_u(5.4.0):c1) Uninverting(_x(5.4.0):c1) Uninverting(_z(5.4.0):c1) Uninverting(_y(5.4.0):c1) Uninverting(_10(5.4.0):c1)))}
   [junit4]   2> 530315 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 530315 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (84)]} 0 74
   [junit4]   2> 530315 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 141
   [junit4]   2> 530318 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@28342326[collection1] realtime
   [junit4]   2> 530319 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-85&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-85)} 0 76
   [junit4]   2> 530319 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@547a9662[collection1] realtime
   [junit4]   2> 530320 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-86&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:11 (-86)} 0 76
   [junit4]   2> 530320 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (87)]} 0 75
   [junit4]   2> 530321 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (90)]} 0 72
   [junit4]   2> 530321 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (92)]} 0 2
   [junit4]   2> 530321 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (89)]} 0 74
   [junit4]   2> 530321 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (88)]} 0 75
   [junit4]   2> 530321 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (91)]} 0 5
   [junit4]   2> 530329 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@744168c4[collection1] realtime
   [junit4]   2> 530330 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (95)]} 0 9
   [junit4]   2> 530330 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[11 (94)]} 0 10
   [junit4]   2> 530330 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-93&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-93)} 0 11
   [junit4]   2> 530331 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (97)]} 0 9
   [junit4]   2> 530336 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1e344836[collection1] realtime
   [junit4]   2> 530336 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-98&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-98)} 0 15
   [junit4]   2> 530337 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-99&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-99)} 0 15
   [junit4]   2> 530337 INFO  (WRITER3) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=96,id=7} DBQs=[DBQ{version=99,q=id:7}, DBQ{version=98,q=id:7}]
   [junit4]   2> 530343 INFO  (WRITER3) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@45bfc898[collection1] realtime
   [junit4]   2> 530344 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (96)]} 0 22
   [junit4]   2> 530344 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (100)]} 0 22
   [junit4]   2> 530344 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (102)]} 0 13
   [junit4]   2> 530344 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (101)]} 0 13
   [junit4]   2> 530351 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@4216073f[collection1] realtime
   [junit4]   2> 530352 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 530352 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-103&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:1 (-103)} 0 20
   [junit4]   2> 530353 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (105)]} 0 15
   [junit4]   2> 530353 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (104)]} 0 16
   [junit4]   2> 530354 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[7 (106)]} 0 10
   [junit4]   2> 530360 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_5,generation=5}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_6,generation=6}
   [junit4]   2> 530361 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 6
   [junit4]   2> 530367 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1d7603a7[collection1] main
   [junit4]   2> 530367 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 530368 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1d7603a7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_12(5.4.0):c11/3:delGen=1) Uninverting(_15(5.4.0):c2/1:delGen=1) Uninverting(_19(5.4.0):c1) Uninverting(_1c(5.4.0):c1) Uninverting(_1b(5.4.0):c1)))}
   [junit4]   2> 530369 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 36
   [junit4]   2> 530370 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6c06f162[collection1] realtime
   [junit4]   2> 530371 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-107&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:7 (-107)} 0 26
   [junit4]   2> 530371 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@53af1add[collection1] realtime
   [junit4]   2> 530372 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-108&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-108)} 0 27
   [junit4]   2> 530372 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (109)]} 0 27
   [junit4]   2> 530373 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (110)]} 0 20
   [junit4]   2> 530426 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7bc5e60c[collection1] realtime
   [junit4]   2> 530427 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-112&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-112)} 0 73
   [junit4]   2> 530427 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 530427 INFO  (WRITER2) [    ] o.a.s.u.UpdateHandler Reordered DBQs detected.  Update=add{_version_=111,id=4} DBQs=[DBQ{version=112,q=id:4}]
   [junit4]   2> 530431 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_6,generation=6}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_7,generation=7}
   [junit4]   2> 530431 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 7
   [junit4]   2> 530433 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@52298efb[collection1] realtime
   [junit4]   2> 530434 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (113)]} 0 79
   [junit4]   2> 530434 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[4 (111)]} 0 80
   [junit4]   2> 530436 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5ea1a720[collection1] main
   [junit4]   2> 530437 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 530437 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (116)]} 0 64
   [junit4]   2> 530437 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (114)]} 0 66
   [junit4]   2> 530437 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (115)]} 0 64
   [junit4]   2> 530437 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5ea1a720[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_12(5.4.0):c11/4:delGen=2) Uninverting(_15(5.4.0):c2/1:delGen=1) Uninverting(_1d(5.4.0):c2/1:delGen=1) Uninverting(_1g(5.4.0):c1)))}
   [junit4]   2> 530438 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 67
   [junit4]   2> 530439 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-117&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-117)} 0 65
   [junit4]   2> 530440 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-118&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:4 (-118)} 0 12
   [junit4]   2> 530440 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (119)]} 0 6
   [junit4]   2> 530454 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@a14d437[collection1] realtime
   [junit4]   2> 530454 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-121&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:6 (-121)} 0 16
   [junit4]   2> 530455 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (122)]} 0 17
   [junit4]   2> 530455 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (123)]} 0 17
   [junit4]   2> 530455 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (120)]} 0 20
   [junit4]   2> 530456 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (125)]} 0 16
   [junit4]   2> 530456 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (124)]} 0 17
   [junit4]   2> 530460 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 530462 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@9a10a6d[collection1] realtime
   [junit4]   2> 530462 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 530462 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-126&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:0 (-126)} 0 22
   [junit4]   2> 530463 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 530463 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[6 (128)]} 0 9
   [junit4]   2> 530463 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (127)]} 0 13
   [junit4]   2> 530466 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 530472 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@3e06f62d[collection1] realtime
   [junit4]   2> 530472 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-129&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-129)} 0 17
   [junit4]   2> 530473 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[0 (130)]} 0 17
   [junit4]   2> 530476 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 530476 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 530476 INFO  (WRITER2) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@1fa88631[collection1] realtime
   [junit4]   2> 530476 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001/]
   [junit4]   2> 530476 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4be82e3a
   [junit4]   2> 530476 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-131&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-131)} 0 21
   [junit4]   2> 530477 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-132&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-132)} 0 21
   [junit4]   2> 530477 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001/index/
   [junit4]   2> 530478 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (133)]} 0 21
   [junit4]   2> 530478 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.search.TestReloadDeadlock_43D7C7E8846AE831-001/init-core-data-001/
   [junit4]   2> 530483 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 530483 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 530483 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 530483 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 530484 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 530484 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 530484 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 530484 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 530484 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 530487 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 530489 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 530490 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 530491 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 530497 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/admin/system,/update/csv,/admin/properties,standard,/admin/segments,/replication,/admin/file,/admin/plugins,/update/json,/admin/mbeans,/schema,/get,/admin/ping,/admin/threads,/config,/update/json/docs,/admin/luke,/admin/logging
   [junit4]   2> 530498 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 530498 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 530499 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 530531 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@7ee10224[collection1] realtime
   [junit4]   2> 530531 INFO  (WRITER5) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 530531 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-134&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-134)} 0 68
   [junit4]   2> 530532 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4f750679[collection1] main
   [junit4]   2> 530532 INFO  (WRITER5) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_7,generation=7}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_8,generation=8}
   [junit4]   2> 530532 INFO  (WRITER5) [    ] o.a.s.c.SolrCore newest commit generation = 8
   [junit4]   2> 530532 WARN  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 530533 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 530533 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 530533 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 530533 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 530534 INFO  (searcherExecutor-2045-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4f750679[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1m(5.4.0):c9/2:delGen=1) Uninverting(_1n(5.4.0):c1) Uninverting(_1o(5.4.0):c1)))}
   [junit4]   2> 530534 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 530536 INFO  (WRITER5) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@491594e[collection1] main
   [junit4]   2> 530536 INFO  (WRITER5) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 530536 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@491594e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1m(5.4.0):c9/3:delGen=1) Uninverting(_1n(5.4.0):c1) Uninverting(_1o(5.4.0):c1) Uninverting(_1p(5.4.0):c1)))}
   [junit4]   2> 530537 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[14 (135)]} 0 72
   [junit4]   2> 530537 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 73
   [junit4]   2> 530538 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2b7a1bdc[collection1] realtime
   [junit4]   2> 530538 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-136&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:14 (-136)} 0 65
   [junit4]   2> 530539 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[10 (137)]} 0 65
   [junit4]   2> 530540 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (138)]} 0 62
   [junit4]   2> 530540 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (139)]} 0 62
   [junit4]   2> 530545 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2d96fd6c[collection1] realtime
   [junit4]   2> 530545 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-140&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:5 (-140)} 0 67
   [junit4]   2> 530546 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (141)]} 0 14
   [junit4]   2> 530546 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 530548 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.VersionInfo Found MAX value 133 from Terms for _version_ in index
   [junit4]   2> 530550 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.UpdateLog Took 4 ms to seed version buckets with highest version 141
   [junit4]   2> 530550 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Creating new IndexWriter...
   [junit4]   2> 530550 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Waiting until IndexWriter is unused... core=collection1
   [junit4]   2> 530550 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Closing old IndexWriter... core=collection1
   [junit4]   2> 530553 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_8,generation=8}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_9,generation=9}
   [junit4]   2> 530553 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 9
   [junit4]   2> 530554 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=23, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7355819685894223]
   [junit4]   2> 530556 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_9,generation=9}
   [junit4]   2> 530556 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 9
   [junit4]   2> 530557 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 530560 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@226779ab[collection1] main
   [junit4]   2> 530560 INFO  (TEST-TestReloadDeadlock.testReloadDeadlock-seed#[43D7C7E8846AE831]) [    x:collection1] o.a.s.c.CoreContainer replacing core: collection1
   [junit4]   2> 530561 INFO  (searcherExecutor-2045-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@226779ab[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1m(5.4.0):c9/3:delGen=2) Uninverting(_1n(5.4.0):c1) Uninverting(_1r(5.4.0):c1) Uninverting(_1s(5.4.0):c1)))}
   [junit4]   2> 530565 INFO  (WRITER6) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@417a8945[collection1] realtime
   [junit4]   2> 530565 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (142)]} 0 28
   [junit4]   2> 530567 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (144)]} 0 27
   [junit4]   2> 530567 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (145)]} 0 27
   [junit4]   2> 530573 INFO  (WRITER7) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@71ca2db3[collection1] realtime
   [junit4]   2> 530573 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-146&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:8 (-146)} 0 33
   [junit4]   2> 530573 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-143&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:2 (-143)} 0 34
   [junit4]   2> 530574 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 530575 INFO  (WRITER0) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_9,generation=9}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@3eb11dcd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e1b9de2),segFN=segments_a,generation=10}
   [junit4]   2> 530575 INFO  (WRITER0) [    ] o.a.s.c.SolrCore newest commit generation = 10
   [junit4]   2> 530575 INFO  (WRITER0) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@2cafa4cc[collection1] main
   [junit4]   2> 530576 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 530576 INFO  (searcherExecutor-2040-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2cafa4cc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1m(5.4.0):c9/5:delGen=3) Uninverting(_1n(5.4.0):c1) Uninverting(_1r(5.4.0):c1) Uninverting(_1u(5.4.0):c1) Uninverting(_1v(5.4.0):c1)))}
   [junit4]   2> 530577 INFO  (WRITER0) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 31
   [junit4]   2> 530577 INFO  (WRITER0) [    ] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@74b2a94f
   [junit4]   2> 530580 INFO  (WRITER1) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@10b8958e[collection1] realtime
   [junit4]   2> 530580 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-148&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:9 (-148)} 0 33
   [junit4]   2> 530581 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[13 (150)]} 0 13
   [junit4]   2> 530581 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[8 (152)]} 0 7
   [junit4]   2> 530582 INFO  (WRITER0) [    ] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=8,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=1,adds=1,deletesById=0,deletesByQuery=1,errors=1,cumulative_adds=88,cumulative_deletesById=0,cumulative_deletesByQuery=54,cumulative_errors=1,transaction_logs_total_size=2582,transaction_logs_total_number=5}
   [junit4]   2> 530582 INFO  (WRITER5) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[5 (147)]} 0 40
   [junit4]   2> 530583 INFO  (WRITER1) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (154)]} 0 2
   [junit4]   2> 530583 INFO  (WRITER6) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (153)]} 0 9
   [junit4]   2> 530583 INFO  (WRITER3) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[12 (149)]} 0 16
   [junit4]   2> 530583 INFO  (WRITER2) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {add=[3 (151)]} 0 15
   [junit4]   2> 530582 INFO  (WRITER0) [    ] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 530593 INFO  (WRITER4) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@12bcb8df[collection1] realtime
   [junit4]   2> 530593 INFO  (WRITER4) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={_version_=-155&update.distrib=FROMLEADER&versions=true&wt=json&indent=true} {deleteByQuery=id:13 (-155)} 0 11
   [junit4]   2> 530595 INFO  (WRITER7) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] web

[...truncated too long message...]

ient.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8284, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37696/m: Could not find collection : awholynewstresscollection_collection0_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([43D7C7E8846AE831]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8289, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37696/m: Could not find collection : awholynewstresscollection_collection5_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([43D7C7E8846AE831]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8288, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37696/m: Could not find collection : awholynewstresscollection_collection4_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([43D7C7E8846AE831]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)
   [junit4]   2> 1541559 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[43D7C7E8846AE831]-worker) [n:127.0.0.1:37696_m c:addReplicaColl s:shard2 r:core_node3 x:addReplicaColl_shard2_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=2, maxDocsPerChunk=106, blockSize=3), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=2, blockSize=3)), sim=DefaultSimilarity, locale=el_GR, timezone=Antarctica/Casey
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=156328080,total=506462208
   [junit4]   2> NOTE: All tests run in this JVM: [TestPivotHelperCode, WordBreakSolrSpellCheckerTest, FacetPivotSmallTest, PKIAuthenticationIntegrationTest, TestSearchPerf, TestCopyFieldCollectionResource, StatelessScriptUpdateProcessorFactoryTest, TestExpandComponent, CloudExitableDirectoryReaderTest, JsonLoaderTest, AtomicUpdatesTest, ChaosMonkeySafeLeaderTest, TestFastWriter, TestCoreContainer, TestManagedResource, LukeRequestHandlerTest, CopyFieldTest, ConvertedLegacyTest, DistributedMLTComponentTest, TestConfigOverlay, UpdateParamsTest, TestMissingGroups, TestRTimerTree, TestNoOpRegenerator, HLLUtilTest, SSLMigrationTest, TestConfigReload, ShardRoutingTest, TestCoreDiscovery, AssignTest, TestMaxScoreQueryParser, BigEndianAscendingWordDeserializerTest, DistributedTermsComponentTest, SuggesterTSTTest, ZkSolrClientTest, TestPKIAuthenticationPlugin, BasicAuthIntegrationTest, RuleEngineTest, SliceStateTest, TestAddFieldRealTimeGet, DistributedFacetPivotSmallTest, TestImplicitCoreProperties, TestSolrCoreProperties, TestReplicaProperties, SpellPossibilityIteratorTest, AlternateDirectoryTest, SystemInfoHandlerTest, TestAuthenticationFramework, TestLMDirichletSimilarityFactory, TestJsonFacets, ZkCLITest, DistanceUnitsTest, DistributedFacetPivotWhiteBoxTest, DirectSolrConnectionTest, TestFuzzyAnalyzedSuggestions, TestManagedSynonymFilterFactory, TestSolrDeletionPolicy2, TestManagedStopFilterFactory, TestFiltering, ResourceLoaderTest, TestJsonRequest, TestReloadDeadlock, TestStressVersions, ZkStateReaderTest, TestCrossCoreJoin, OpenCloseCoreStressTest, TestSolrConfigHandler, TestDistributedGrouping, RequiredFieldsTest, ClusterStateUpdateTest, TestManagedResourceStorage, TestCustomSort, SaslZkACLProviderTest, OutOfBoxZkACLAndCredentialsProvidersTest, DocValuesMultiTest, IndexSchemaRuntimeFieldTest, TestMacros, CollectionsAPIDistributedZkTest]
   [junit4] Completed [221/533] on J1 in 297.01s, 1 test, 1 error <<< FAILURES!

[...truncated 996 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 533 suites, 2110 tests, 2 errors, 26 ignored (14 assumptions)

Total time: 174 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1197246708 bytes
Compression is 0.0%
Took 3 min 18 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 924 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/924/

6 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at https://127.0.0.1:40539: Cannot create collection halfcollection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 1. This allows a maximum of 1 to be created. Value of numShards is 2 and value of replicationFactor is 1. This requires 2 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:40539: Cannot create collection halfcollection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 1. This allows a maximum of 1 to be created. Value of numShards is 2 and value of replicationFactor is 1. This requires 2 shards to be created (higher than the allowed number)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:301)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:449)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:168)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:36786/collection2

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36786/collection2
	at __randomizedtesting.SeedInfo.seed([5419518DC5D98BED:DC4D6E576B25E615]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:570)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:483)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:464)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1503)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:665)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:153)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 54 more


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1439192280794,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Direct_0.doc, _3_Direct_0.tim, _3_Direct_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Direct_0.doc, _4_Direct_0.tim, _4_Direct_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Direct_0.doc, _5_Direct_0.tim, _5_Direct_0.tip, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Direct_0.doc, _6_Direct_0.tim, _6_Direct_0.tip, segments_2]}]> but was:<[{indexVersion=1439192280794,generation=3,filelist=[_4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Direct_0.doc, _4_Direct_0.tim, _4_Direct_0.tip, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_Direct_0.doc, _7_Direct_0.tim, _7_Direct_0.tip, segments_3]}, {indexVersion=1439192280794,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Direct_0.doc, _3_Direct_0.tim, _3_Direct_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Direct_0.doc, _4_Direct_0.tim, _4_Direct_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Direct_0.doc, _5_Direct_0.tim, _5_Direct_0.tip, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Direct_0.doc, _6_Direct_0.tim, _6_Direct_0.tip, segments_2]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1439192280794,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Direct_0.doc, _3_Direct_0.tim, _3_Direct_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Direct_0.doc, _4_Direct_0.tim, _4_Direct_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Direct_0.doc, _5_Direct_0.tim, _5_Direct_0.tip, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Direct_0.doc, _6_Direct_0.tim, _6_Direct_0.tip, segments_2]}]> but was:<[{indexVersion=1439192280794,generation=3,filelist=[_4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Direct_0.doc, _4_Direct_0.tim, _4_Direct_0.tip, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_Direct_0.doc, _7_Direct_0.tim, _7_Direct_0.tip, segments_3]}, {indexVersion=1439192280794,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Direct_0.doc, _3_Direct_0.tim, _3_Direct_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Direct_0.doc, _4_Direct_0.tim, _4_Direct_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Direct_0.doc, _5_Direct_0.tim, _5_Direct_0.tip, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Direct_0.doc, _6_Direct_0.tim, _6_Direct_0.tip, segments_2]}]>
	at __randomizedtesting.SeedInfo.seed([5419518DC5D98BED:71CE4ABDB59185EE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.search.TestReloadDeadlock.testReloadDeadlock

Error Message:
Captured an uncaught exception in thread: Thread[id=29010, name=WRITER6, state=RUNNABLE, group=TGRP-TestReloadDeadlock]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=29010, name=WRITER6, state=RUNNABLE, group=TGRP-TestReloadDeadlock]
Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
	at __randomizedtesting.SeedInfo.seed([5419518DC5D98BED]:0)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1662)
	at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1519)
	at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:201)
	at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:778)
	at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:194)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1089)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706)
	at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
	at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
	at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
	at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
	at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
	at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
	at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
	at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
Caused by: java.lang.NullPointerException
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1631)
	... 21 more


FAILED:  junit.framework.TestSuite.org.apache.solr.search.TestSimpleQParserPlugin

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [TransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [TransactionLog]
	at __randomizedtesting.SeedInfo.seed([5419518DC5D98BED]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:236)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=26350, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=26350, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:54930/sf_kf/fq: Could not find collection : awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([5419518DC5D98BED]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10869 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/init-core-data-001
   [junit4]   2> 1012579 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[5419518DC5D98BED]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1012583 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1012584 INFO  (Thread-3252) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1012584 INFO  (Thread-3252) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1012684 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.ZkTestServer start zk server on port:54420
   [junit4]   2> 1012684 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1012685 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1012687 INFO  (zkCallback-593-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19d28999 name:ZooKeeperConnection Watcher:127.0.0.1:54420 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1012688 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1012688 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1012688 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1012690 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1012691 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1012692 INFO  (zkCallback-594-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d26a2cc name:ZooKeeperConnection Watcher:127.0.0.1:54420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1012692 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1012692 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1012692 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1012694 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1012696 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1012697 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1012710 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1012710 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1012713 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1012714 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1012716 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1012717 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1012718 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1012718 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1012720 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1012720 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1012722 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1012722 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1012724 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1012724 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1012726 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1012726 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1012733 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1012734 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1012735 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1012736 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1012737 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1012737 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1013014 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1
   [junit4]   2> 1013015 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1013020 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1579a92d{/,null,AVAILABLE}
   [junit4]   2> 1013021 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.ServerConnector Started ServerConnector@64d9bcd{HTTP/1.1}{127.0.0.1:59397}
   [junit4]   2> 1013021 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.Server Started @1017295ms
   [junit4]   2> 1013021 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores, hostContext=/, hostPort=59397, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/tempDir-001/control/data}
   [junit4]   2> 1013022 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1013022 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/'
   [junit4]   2> 1013038 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1013039 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1013040 INFO  (zkCallback-595-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@597c38f name:ZooKeeperConnection Watcher:127.0.0.1:54420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1013040 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1013041 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1013042 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1013042 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/solr.xml
   [junit4]   2> 1013056 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores
   [junit4]   2> 1013056 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.CoreContainer New CoreContainer 124068089
   [junit4]   2> 1013057 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/]
   [junit4]   2> 1013057 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/lib
   [junit4]   2> 1013057 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/lib).
   [junit4]   2> 1013065 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1013066 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1013067 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1013067 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1013067 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54420/solr
   [junit4]   2> 1013067 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1013067 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1013068 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1013069 INFO  (zkCallback-597-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36acf2fb name:ZooKeeperConnection Watcher:127.0.0.1:54420 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1013070 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1013070 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1013071 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1013073 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70488210 name:ZooKeeperConnection Watcher:127.0.0.1:54420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1013073 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1013074 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1013076 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1013079 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1013080 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1013082 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1013084 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1013086 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1013087 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1013088 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1013089 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59397_
   [junit4]   2> 1013090 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59397_
   [junit4]   2> 1013091 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1013092 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1013094 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1013094 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94318785306034180-127.0.0.1:59397_-n_0000000000
   [junit4]   2> 1013095 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59397_
   [junit4]   2> 1013095 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1013096 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.Overseer Overseer (id=94318785306034180-127.0.0.1:59397_-n_0000000000) starting
   [junit4]   2> 1013098 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1013104 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1013105 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1013105 INFO  (OverseerCollectionProcessor-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.OverseerProcessor Process current queue of overseer operations
   [junit4]   2> 1013105 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1013129 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1013129 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1013130 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores
   [junit4]   2> 1013131 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, shard=, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/, collection=control_collection, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1, loadOnStartup=true, dataDir=data/, coreNodeName=, configSetProperties=configsetprops.json, schema=schema.xml}
   [junit4]   2> 1013131 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/
   [junit4]   2> 1013131 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1013132 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1013132 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1013133 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1013135 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1013135 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1013135 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [n:127.0.0.1:59397_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1013136 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:59397_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59397",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 1013136 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:59397_",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59397",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1013136 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1013137 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1013138 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1013145 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 1014133 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1014133 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1014134 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1014134 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1014136 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1014136 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/'
   [junit4]   2> 1014150 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1014157 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1014169 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1014199 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1014200 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1014208 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1014416 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1014417 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1014418 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1014422 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1014443 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1014443 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1014443 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1014444 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1014444 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@49edeec0
   [junit4]   2> 1014445 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/data
   [junit4]   2> 1014445 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1014445 WARN  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1014446 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/data/index
   [junit4]   2> 1014446 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=29, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3337255747955862]
   [junit4]   2> 1014447 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@1c9760e1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@16596271),segFN=segments_1,generation=1}
   [junit4]   2> 1014447 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1014448 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/data/
   [junit4]   2> 1014448 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1014453 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1014453 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1014453 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1014454 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1014454 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1014454 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1014454 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1014454 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1014454 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1014459 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1014460 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1014461 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1014462 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1014466 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/system,/admin/logging,/admin/properties,/admin/file,/schema,/update/json/docs,standard,/admin/threads,/admin/mbeans,/update,/admin/luke,/replication,/config,/admin/segments,/update/csv,/update/json,/admin/plugins,/get,/admin/ping
   [junit4]   2> 1014467 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1014468 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1014468 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1014469 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1014469 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1014469 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=4.3076171875, floorSegmentMB=1.1005859375, forceMergeDeletesPctAllowed=10.28177578399201, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1014470 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@1c9760e1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@16596271),segFN=segments_1,generation=1}
   [junit4]   2> 1014470 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1014470 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@335ae880[collection1] main
   [junit4]   2> 1014470 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1014471 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1014471 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1014471 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1014472 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1014472 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1014472 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1014472 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1014472 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1014473 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1014473 INFO  (searcherExecutor-2574-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@335ae880[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1014473 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1014474 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1014474 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1014474 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1509100566877306880
   [junit4]   2> 1014476 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1509100566877306880
   [junit4]   2> 1014476 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1014477 INFO  (coreLoadExecutor-2573-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1014477 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59397 collection:control_collection shard:shard1
   [junit4]   2> 1014478 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1014483 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94318785306034180-core_node1-n_0000000000
   [junit4]   2> 1014484 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1014486 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1014487 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1014487 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1014487 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59397/collection1/
   [junit4]   2> 1014487 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1014487 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59397/collection1/ has no replicas
   [junit4]   2> 1014488 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59397/collection1/ shard1
   [junit4]   2> 1014488 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1014487 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1014493 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1014494 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59397",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1014596 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1014598 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 1014637 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1014638 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1014639 INFO  (zkCallback-600-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55be094 name:ZooKeeperConnection Watcher:127.0.0.1:54420/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1014640 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1014640 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1014640 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1014642 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1014644 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:59397/collection1/ and leader is http://127.0.0.1:59397/collection1/
   [junit4]   2> 1014644 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59397
   [junit4]   2> 1014644 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1014645 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1014645 INFO  (coreZkRegister-2567-thread-1-processing-x:collection1 r:core_node1 n:127.0.0.1:59397_ s:shard1 c:control_collection) [n:127.0.0.1:59397_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1014650 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1014651 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:59397_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59397",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"} current state version: 2
   [junit4]   2> 1014652 INFO  (OverseerStateUpdate-94318785306034180-127.0.0.1:59397_-n_0000000000) [n:127.0.0.1:59397_    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:59397_",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59397",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1014755 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1014755 INFO  (zkCallback-600-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1014758 INFO  (zkCallback-600-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1014759 INFO  (zkCallback-598-thread-1-processing-n:127.0.0.1:59397_) [n:127.0.0.1:59397_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1014936 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/shard-1-001/cores/collection1
   [junit4]   2> 1014937 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/shard-1-001
   [junit4]   2> 1014938 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1014940 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65a7e1a0{/,null,AVAILABLE}
   [junit4]   2> 1014940 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6548c750{HTTP/1.1}{127.0.0.1:44057}
   [junit4]   2> 1014940 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.e.j.s.Server Started @1019214ms
   [junit4]   2> 1014940 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=44057, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/shard-1-001/cores}
   [junit4]   2> 1014941 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1014941 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_5419518DC5D98BED-001/shard-1-001/'
   [junit4]   2> 1014958 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1014959 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[5419518DC5D98BED]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1014960 INFO  

[...truncated too long message...]

unit4]   2> 4863425 INFO  (SUITE-TestReloadDeadlock-seed#[5419518DC5D98BED]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 4863432 INFO  (SUITE-TestReloadDeadlock-seed#[5419518DC5D98BED]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 4863432 INFO  (SUITE-TestReloadDeadlock-seed#[5419518DC5D98BED]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestReloadDeadlock_5419518DC5D98BED-001/init-core-data-001/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestReloadDeadlock_5419518DC5D98BED-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 4863432 INFO  (SUITE-TestReloadDeadlock-seed#[5419518DC5D98BED]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestReloadDeadlock_5419518DC5D98BED-001/init-core-data-001/index
   [junit4]   2> 4863433 INFO  (SUITE-TestReloadDeadlock-seed#[5419518DC5D98BED]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestReloadDeadlock_5419518DC5D98BED-001/init-core-data-001 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestReloadDeadlock_5419518DC5D98BED-001/init-core-data-001;done=false>>]
   [junit4]   2> 4863433 INFO  (SUITE-TestReloadDeadlock-seed#[5419518DC5D98BED]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestReloadDeadlock_5419518DC5D98BED-001/init-core-data-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {_version_=Lucene50(blocksize=128), val_l=PostingsFormat(name=LuceneFixedGap), id=BlockTreeOrds(blocksize=128)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=et, timezone=America/Sao_Paulo
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=135453208,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [TestDocSet, SmileWriterTest, SampleTest, TestCollationField, MoreLikeThisHandlerTest, TestStressRecovery, TestConfigSetProperties, CursorMarkTest, StatsComponentTest, XsltUpdateRequestHandlerTest, TestCloudManagedSchema, TestBinaryResponseWriter, DistributedExpandComponentTest, RecoveryAfterSoftCommitTest, PrimUtilsTest, TestStandardQParsers, ActionThrottleTest, TestStressReorder, QueryEqualityTest, TestSolr4Spatial2, AsyncCallRequestStatusResponseTest, TestDistributedMissingSort, TestExceedMaxTermLength, TestDownShardTolerantSearch, TestSimpleTrackingShardHandler, TestReload, AsyncMigrateRouteKeyTest, TestTrie, LoggingHandlerTest, TestGroupingSearch, RequestLoggingTest, TestSmileRequest, TestSolrJ, TestPseudoReturnFields, SpatialRPTFieldTypeTest, TestXIncludeConfig, TestManagedSchemaFieldTypeResource, CircularListTest, TestCSVResponseWriter, TestRuleBasedAuthorizationPlugin, TestSortByMinMaxFunction, TestExtendedDismaxParser, TestQuerySenderNoQuery, SecurityConfHandlerTest, OpenExchangeRatesOrgProviderTest, RecoveryZkTest, NumericFieldsTest, TestJsonFacets, TestShortCircuitedRequests, TestRTimer, OverriddenZkACLAndCredentialsProvidersTest, TestBulkSchemaAPI, TestQueryUtils, PreAnalyzedUpdateProcessorTest, SpellCheckCollatorTest, DocValuesTest, TestTrieFacet, DistributedQueueTest, SoftAutoCommitTest, HdfsLockFactoryTest, TestDynamicFieldResource, PathHierarchyTokenizerFactoryTest, ZkStateWriterTest, TestCloudSchemaless, AutoCommitTest, SynonymTokenizerTest, TestManagedSchemaDynamicFieldResource, ResponseLogComponentTest, HdfsWriteToMultipleCollectionsTest, DistributedFacetPivotLongTailTest, TestInitParams, DistributedVersionInfoTest, SearchHandlerTest, OverseerCollectionProcessorTest, TestRandomMergePolicy, HardAutoCommitTest, TestAuthenticationFramework, TestHashQParserPlugin, TestRecovery, TestRestManager, TestInitQParser, FullSolrCloudDistribCmdsTest, DOMUtilTest, TestSimpleQParserPlugin, TestSystemIdResolver, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, OpenCloseCoreStressTest, BasicZkTest, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedGrouping, TermVectorComponentDistributedTest, TestRealTimeGet, TestStressVersions, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, SolrCmdDistributorTest, PeerSyncTest, BadIndexSchemaTest, TestSort, TestFiltering, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, TestIndexSearcher, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SuggesterFSTTest, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, TestRemoteStreaming, TestSolrDeletionPolicy1, SolrInfoMBeanTest, TestSurroundQueryParser, LukeRequestHandlerTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, TestQueryTypes, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestIndexingPerformance, TestSolrQueryParser, TestArbitraryIndexDir, FastVectorHighlighterTest, RegexBoostProcessorTest, IndexSchemaTest, ReturnFieldsTest, MBeansHandlerTest, QueryParsingTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestLFUCache, SOLR749Test, AlternateDirectoryTest, TestSolrIndexConfig, TestBinaryField, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestCharFilters, TestCodecSupport, EchoParamsTest, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, PluginInfoTest, TestFastLRUCache, DateMathParserTest, PreAnalyzedFieldTest, TestSuggestSpellingConverter, RAMDirectoryFactoryTest, ClusterStateTest, TestUtils, ZkNodePropsTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, CursorPagingTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestHighlightDedupGrouping, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, AliasIntegrationTest, CollectionReloadTest, CollectionStateFormat2Test, ConnectionManagerTest, CustomCollectionTest, DeleteInactiveReplicaTest, DistribCursorPagingTest, LeaderFailoverAfterPartitionTest, OverseerRolesTest, ShardSplitTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, SolrXmlInZkTest, TestCloudPivotFacet, TestCollectionAPI, TestMiniSolrCloudCluster, TestSolrCloudWithKerberosAlt, HdfsBasicDistributedZk2Test, HdfsCollectionsAPIDistributedZkTest, TestReloadDeadlock]
   [junit4] Completed [496/532] on J2 in 1.42s, 1 test, 1 error <<< FAILURES!

[...truncated 109 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150810_025032_632.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid32215.hprof ...
   [junit4] Heap dump file created [514097695 bytes in 3.881 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 15 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 532 suites, 2106 tests, 1 suite-level error, 4 errors, 1 failure, 27 ignored (14 assumptions)

Total time: 236 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1748536180 bytes
Compression is 0.0%
Took 5 min 8 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 923 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/923/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:42293/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:42293/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([422BF5155421CCA:8C76808BFBBE7132]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:332)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.TestRandomRequestDistribution.testRequestTracking

Error Message:
Shard a1x2_shard1_replica1 received all 10 requests

Stack Trace:
java.lang.AssertionError: Shard a1x2_shard1_replica1 received all 10 requests
	at __randomizedtesting.SeedInfo.seed([422BF5155421CCA:4C1EE691A1490D5C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.TestRandomRequestDistribution.testRequestTracking(TestRandomRequestDistribution.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.search.TestSolrQueryParser.testFilter

Error Message:
Query SortedIntDocSetTopFilter does not implement createWeight

Stack Trace:
java.lang.UnsupportedOperationException: Query SortedIntDocSetTopFilter does not implement createWeight
	at __randomizedtesting.SeedInfo.seed([422BF5155421CCA:CCF0DAC7CA4B3B90]:0)
	at org.apache.lucene.search.Query.createWeight(Query.java:79)
	at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:855)
	at org.apache.lucene.search.ConstantScoreQuery.createWeight(ConstantScoreQuery.java:117)
	at org.apache.solr.query.FilterQuery.createWeight(FilterQuery.java:96)
	at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:855)
	at org.apache.lucene.search.BooleanWeight.<init>(BooleanWeight.java:56)
	at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:203)
	at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:855)
	at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:838)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:486)
	at org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1259)
	at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:941)
	at org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1103)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1625)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1501)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:555)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:522)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:320)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:302)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:831)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:800)
	at org.apache.solr.search.TestSolrQueryParser.testFilter(TestSolrQueryParser.java:224)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=13344, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13344, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48006/qa_: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([422BF5155421CCA]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10791 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestRandomRequestDistribution
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/init-core-data-001
   [junit4]   2> 523209 INFO  (SUITE-TestRandomRequestDistribution-seed#[422BF5155421CCA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 523213 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 523213 INFO  (Thread-5111) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 523214 INFO  (Thread-5111) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 523313 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.ZkTestServer start zk server on port:57876
   [junit4]   2> 523314 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 523315 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 523319 INFO  (zkCallback-410-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3700c5d9 name:ZooKeeperConnection Watcher:127.0.0.1:57876 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 523319 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 523320 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 523320 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 523323 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 523324 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 523326 INFO  (zkCallback-411-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42a014bc name:ZooKeeperConnection Watcher:127.0.0.1:57876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 523327 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 523327 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 523327 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 523330 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 523332 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 523334 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 523336 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 523336 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 523340 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 523340 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 523342 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 523343 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 523345 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 523345 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 523347 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 523347 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 523349 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 523350 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 523352 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 523352 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 523354 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 523355 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 523357 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 523358 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 523360 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 523361 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 523363 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 523363 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 523628 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1
   [junit4]   2> 523630 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 523631 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e4a16f0{/,null,AVAILABLE}
   [junit4]   2> 523633 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7d8dec9e{HTTP/1.1}{127.0.0.1:51063}
   [junit4]   2> 523634 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.Server Started @526660ms
   [junit4]   2> 523634 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=51063, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/tempDir-001/control/data, hostContext=/}
   [junit4]   2> 523634 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 523634 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/'
   [junit4]   2> 523649 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 523650 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 523651 INFO  (zkCallback-412-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a374f0b name:ZooKeeperConnection Watcher:127.0.0.1:57876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 523652 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 523652 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 523653 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 523653 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/solr.xml
   [junit4]   2> 523667 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores
   [junit4]   2> 523667 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoreContainer New CoreContainer 291729166
   [junit4]   2> 523667 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/]
   [junit4]   2> 523667 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/lib
   [junit4]   2> 523668 WARN  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/lib).
   [junit4]   2> 523676 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 523677 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 523678 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 523678 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 523678 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57876/solr
   [junit4]   2> 523678 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 523679 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 523679 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 523680 INFO  (zkCallback-414-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ae7f2fa name:ZooKeeperConnection Watcher:127.0.0.1:57876 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 523681 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 523682 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 523683 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 523684 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@163814db name:ZooKeeperConnection Watcher:127.0.0.1:57876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 523685 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 523686 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 523688 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 523690 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 523692 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 523694 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 523696 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 523699 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 523701 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 523702 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 523703 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51063_
   [junit4]   2> 523704 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51063_
   [junit4]   2> 523705 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 523706 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 523708 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 523708 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94305808602955780-127.0.0.1:51063_-n_0000000000
   [junit4]   2> 523709 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51063_
   [junit4]   2> 523709 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 523710 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer Overseer (id=94305808602955780-127.0.0.1:51063_-n_0000000000) starting
   [junit4]   2> 523712 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 523718 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 523719 INFO  (OverseerCollectionProcessor-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 523719 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 523719 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 523763 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 523763 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 523764 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores
   [junit4]   2> 523765 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, shard=, name=collection1, dataDir=data/, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1, configSetProperties=configsetprops.json, loadOnStartup=true, collection=control_collection, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/, config=solrconfig.xml}
   [junit4]   2> 523765 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/
   [junit4]   2> 523765 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 523766 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 523766 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 523766 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:51063_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 523766 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 523767 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 523767 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 523769 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51063",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:51063_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 523770 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51063",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:51063_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 523770 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 523770 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 523772 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 523774 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 524768 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 524768 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 524769 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 524769 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 524770 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 524770 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/'
   [junit4]   2> 524786 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 524792 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 524805 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 524849 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 524850 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 524857 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 525095 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 525103 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 525105 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 525114 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 525117 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 525119 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 525120 WARN  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 525121 WARN  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 525121 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 525121 WARN  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 525122 WARN  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 525123 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 525123 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 525123 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 525123 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 525123 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e2fce9c
   [junit4]   2> 525124 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/data
   [junit4]   2> 525124 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/data/index/
   [junit4]   2> 525124 WARN  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 525130 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/data/index
   [junit4]   2> 525131 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.24973780600195986]
   [junit4]   2> 525133 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ad54c5d),segFN=segments_1,generation=1}
   [junit4]   2> 525133 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 525134 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/data/
   [junit4]   2> 525134 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 525139 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 525139 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 525140 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 525140 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 525140 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 525140 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 525140 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 525141 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 525141 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 525144 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 525145 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 525146 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 525148 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 525151 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/schema,/config,/admin/system,standard,/update/csv,/update/json,/admin/file,/admin/luke,/admin/threads,/admin/properties,/admin/mbeans,/admin/logging,/admin/ping,/admin/plugins,/update,/get,/update/json/docs,/admin/segments
   [junit4]   2> 525153 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 525153 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 525153 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 525154 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 525154 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 525155 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=2.994140625, floorSegmentMB=1.4267578125, forceMergeDeletesPctAllowed=28.76103670992622, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4982097479023936
   [junit4]   2> 525156 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ad54c5d),segFN=segments_1,generation=1}
   [junit4]   2> 525156 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 525156 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2af3a891[collection1] main
   [junit4]   2> 525157 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 525157 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 525158 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 525158 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 525158 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 525158 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 525159 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 525159 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 525159 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 525159 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 525160 INFO  (searcherExecutor-1543-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2af3a891[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 525160 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 525160 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 525160 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 525160 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1508892939685724160
   [junit4]   2> 525163 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1508892939685724160
   [junit4]   2> 525163 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 525163 INFO  (coreLoadExecutor-1542-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 525164 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:51063 collection:control_collection shard:shard1
   [junit4]   2> 525164 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 525168 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94305808602955780-core_node1-n_0000000000
   [junit4]   2> 525169 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 525170 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 525170 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 525170 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 525171 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51063/collection1/
   [junit4]   2> 525171 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 525171 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 525171 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51063/collection1/ has no replicas
   [junit4]   2> 525171 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51063/collection1/ shard1
   [junit4]   2> 525171 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 525175 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 525176 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51063",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 525268 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 525269 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 525271 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@164891e name:ZooKeeperConnection Watcher:127.0.0.1:57876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 525271 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 525271 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 525271 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 525275 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 525275 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 525275 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 525279 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 525280 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 525287 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 525287 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 525291 INFO  (zkCallback-418-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@180877a1 name:ZooKeeperConnection Watcher:127.0.0.1:57876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 525292 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 525292 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 525293 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 525294 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 525294 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 525295 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 525326 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:51063/collection1/ and leader is http://127.0.0.1:51063/collection1/
   [junit4]   2> 525326 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51063
   [junit4]   2> 525327 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 525327 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 525327 INFO  (coreZkRegister-1536-thread-1-processing-r:core_node1 n:127.0.0.1:51063_ s:shard1 x:collection1 c:control_collection) [n:127.0.0.1:51063_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 525328 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 525329 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51063",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:51063_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection"} current state version: 2
   [junit4]   2> 525329 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51063",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:51063_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 525330 INFO  (OverseerStateUpdate-94305808602955780-127.0.0.1:51063_-n_0000000000) [n:127.0.0.1:51063_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 525331 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 525331 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 525332 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 525333 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 525435 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 525435 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 525437 INFO  (zkCallback-415-thread-1-processing-n:127.0.0.1:51063_) [n:127.0.0.1:51063_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 525437 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 525576 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/cores/collection1
   [junit4]   2> 525577 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001
   [junit4]   2> 525578 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 525582 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@104ad00c{/,null,AVAILABLE}
   [junit4]   2> 525583 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@75bd31e1{HTTP/1.1}{127.0.0.1:60221}
   [junit4]   2> 525583 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.e.j.s.Server Started @528609ms
   [junit4]   2> 525583 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, hostPort=60221, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/cores}
   [junit4]   2> 525584 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 525584 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/'
   [junit4]   2> 525599 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 525600 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 525601 INFO  (zkCallback-419-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@283e5791 name:ZooKeeperConnection Watcher:127.0.0.1:57876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 525601 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 525602 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 525603 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 525603 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/solr.xml
   [junit4]   2> 525617 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/cores
   [junit4]   2> 525617 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoreContainer New CoreContainer 1515996072
   [junit4]   2> 525617 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/]
   [junit4]   2> 525617 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/lib
   [junit4]   2> 525618 WARN  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomRequestDistribution_422BF5155421CCA-001/shard-1-001/lib).
   [junit4]   2> 525625 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 525626 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 525627 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 525627 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 525627 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57876/solr
   [junit4]   2> 525627 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 525628 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 525628 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 525629 INFO  (zkCallback-421-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37bd2de9 name:ZooKeeperConnection Watcher:127.0.0.1:57876 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 525630 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 525630 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 525635 INFO  (TEST-TestRandomRequestDistribution.testRequestTracking-seed#[422BF5155421CCA]) [n:127.0.0.1:60221_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 525635 INFO  (zkCallback-422-thread-1-processing-n:127.0.0.1:60221_) [n:127.0.0.1:60221_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f822b80 name:ZooKeeperConnection Watcher:127.0.0.1:57876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 525635 INFO  (TEST-TestRandomRequestDistribution.testRequestTra

[...truncated too long message...]

stNestedQueryModifiers
   [junit4]   2> 2257909 INFO  (TEST-TestSolrQueryParser.testSolr4121-seed#[422BF5155421CCA]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSolr4121
   [junit4]   2> 2257910 INFO  (TEST-TestSolrQueryParser.testSolr4121-seed#[422BF5155421CCA]) [    ] o.a.s.c.S.Request [collection1] webapp=null path=null params={indent=true&q=eee_s:'balance'&wt=xml} hits=1 status=0 QTime=0 
   [junit4]   2> 2257910 INFO  (TEST-TestSolrQueryParser.testSolr4121-seed#[422BF5155421CCA]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSolr4121
   [junit4]   2> 2257910 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2257911 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2105978137
   [junit4]   2> 2257911 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1c036d6d
   [junit4]   2> 2257911 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=7,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2257911 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2257912 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2257912 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2257912 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2257923 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2257924 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestSolrQueryParser_422BF5155421CCA-001/init-core-data-001/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestSolrQueryParser_422BF5155421CCA-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 2257924 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestSolrQueryParser_422BF5155421CCA-001/init-core-data-001/index
   [junit4]   2> 2257924 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestSolrQueryParser_422BF5155421CCA-001/init-core-data-001 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestSolrQueryParser_422BF5155421CCA-001/init-core-data-001;done=false>>]
   [junit4]   2> 2257924 INFO  (SUITE-TestSolrQueryParser-seed#[422BF5155421CCA]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestSolrQueryParser_422BF5155421CCA-001/init-core-data-001
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.TestSolrQueryParser_422BF5155421CCA-001
   [junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=30115, maxDocsPerChunk=10, blockSize=1), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=30115, blockSize=1)), sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_OM, timezone=MST
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=192579584,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [BasicDistributedZk2Test, TestCopyFieldCollectionResource, TestFreeTextSuggestions, SharedFSAutoReplicaFailoverUtilsTest, RegexBoostProcessorTest, TestSolrCLIRunExample, TestScoreJoinQPScore, TestQueryTypes, TestQuerySenderNoQuery, TestRemoveLastDynamicCopyField, CoreAdminCreateDiscoverTest, TestSchemaManager, TestSimpleTrackingShardHandler, DistributedIntervalFacetingTest, SortByFunctionTest, CustomCollectionTest, TestRecovery, DOMUtilTest, TestReplicaProperties, TestConfigSetImmutable, DirectUpdateHandlerOptimizeTest, TestMaxScoreQueryParser, TestTrie, TestPerFieldSimilarity, SSLMigrationTest, HardAutoCommitTest, PrimUtilsTest, ResponseHeaderTest, FacetPivotSmallTest, TestTrackingShardHandlerFactory, TestXIncludeConfig, TestUtils, TestIntervalFaceting, AssignTest, DataDrivenBlockJoinTest, HLLSerializationTest, TestBulkSchemaConcurrent, TestLuceneMatchVersion, TestDefaultSimilarityFactory, TestFastOutputStream, SOLR749Test, BadCopyFieldTest, LoggingHandlerTest, SparseHLLTest, DateRangeFieldTest, TestSolrXml, LeaderElectionTest, TestDefaultStatsCache, TestSystemIdResolver, DistributedFacetPivotLargeTest, DistributedDebugComponentTest, TestSort, TestSurroundQueryParser, TestComplexPhraseQParserPlugin, TestSortingResponseWriter, TestRandomRequestDistribution, DistributedQueryElevationComponentTest, TestFieldTypeCollectionResource, TestStressReorder, BlockDirectoryTest, TestDocSet, TestLeaderElectionZkExpiry, BufferStoreTest, TestBinaryResponseWriter, SolrIndexSplitterTest, SpatialHeatmapFacetsTest, PolyFieldTest, MoreLikeThisHandlerTest, TestPartialUpdateDeduplication, OverseerStatusTest, TestRequestStatusCollectionAPI, HighlighterMaxOffsetTest, TestReload, QueryEqualityTest, TestSchemaNameResource, OverseerCollectionProcessorTest, FullSolrCloudDistribCmdsTest, TestClusterStateMutator, TestCSVResponseWriter, BadIndexSchemaTest, TestAuthorizationFramework, TestConfigSets, InfoHandlerTest, TestZkChroot, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, OverseerTest, LeaderElectionIntegrationTest, RecoveryZkTest, ClusterStateUpdateTest, TestRandomFaceting, ShardRoutingCustomTest, TestDistributedSearch, AutoCommitTest, TestFaceting, TestHashPartitioner, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestJoin, DistributedTermsComponentTest, SimpleFacetsTest, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, QueryElevationComponentTest, PeerSyncTest, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, SimplePostToolTest, TestCoreDiscovery, TestExtendedDismaxParser, SpellCheckCollatorTest, SolrRequestParserTest, DocValuesTest, SuggesterTSTTest, SuggesterTest, TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, DistanceFunctionTest, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestQueryUtils, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, TestSolrQueryParser]
   [junit4] Completed [293/531] on J2 in 0.98s, 8 tests, 1 error <<< FAILURES!

[...truncated 769 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 531 suites, 2105 tests, 2 errors, 2 failures, 26 ignored (14 assumptions)

Total time: 219 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1263791021 bytes
Compression is 0.0%
Took 3 min 24 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 922 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/922/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:42445/bpku: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42445/bpku: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:301)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:418)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:168)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2136, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2136, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38271: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([326DF0DCFDA5D52F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10971 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/init-core-data-001
   [junit4]   2> 37293 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 37293 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 38368 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 39606 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 39891 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 39901 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 39991 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 40044 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_42127_hdfs____ggczvm/webapp
   [junit4]   2> 40259 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 41646 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42127
   [junit4]   2> 43365 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 43376 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 43390 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_44285_datanode____940u52/webapp
   [junit4]   2> 43545 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 44482 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44285
   [junit4]   2> 44718 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 44721 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 44740 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34173_datanode____.k4w6rx/webapp
   [junit4]   2> 44864 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 45484 INFO  (IPC Server handler 2 on 35889) [    ] BlockStateChange BLOCK* processReport: from storage DS-dc79f690-45c5-4341-9e7b-d21a9ecfdc1e node DatanodeRegistration(127.0.0.1, datanodeUuid=2ab1eb40-7d62-4b7d-8bbb-652ef08e120e, infoPort=44285, ipcPort=49112, storageInfo=lv=-56;cid=testClusterID;nsid=1912781189;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 45486 INFO  (IPC Server handler 2 on 35889) [    ] BlockStateChange BLOCK* processReport: from storage DS-4a2d38ca-be3a-475a-a6fe-9f190c803e39 node DatanodeRegistration(127.0.0.1, datanodeUuid=2ab1eb40-7d62-4b7d-8bbb-652ef08e120e, infoPort=44285, ipcPort=49112, storageInfo=lv=-56;cid=testClusterID;nsid=1912781189;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 45565 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34173
   [junit4]   2> 45790 INFO  (IPC Server handler 6 on 35889) [    ] BlockStateChange BLOCK* processReport: from storage DS-1238ae68-f13c-45ef-8fad-45ae74939237 node DatanodeRegistration(127.0.0.1, datanodeUuid=f35f67d3-f1b3-4176-8ca5-97e8e924ee6a, infoPort=34173, ipcPort=49113, storageInfo=lv=-56;cid=testClusterID;nsid=1912781189;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 45791 INFO  (IPC Server handler 6 on 35889) [    ] BlockStateChange BLOCK* processReport: from storage DS-1b388443-b697-480c-9ca8-21c76cb2502b node DatanodeRegistration(127.0.0.1, datanodeUuid=f35f67d3-f1b3-4176-8ca5-97e8e924ee6a, infoPort=34173, ipcPort=49113, storageInfo=lv=-56;cid=testClusterID;nsid=1912781189;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 46548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 46550 INFO  (Thread-334) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 46551 INFO  (Thread-334) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 46650 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.ZkTestServer start zk server on port:55770
   [junit4]   2> 46668 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 46711 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 46748 INFO  (zkCallback-16-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48521e1a name:ZooKeeperConnection Watcher:127.0.0.1:55770 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 46749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 46749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 46752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 46780 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 46781 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14f092ad7120000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 46781 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 46784 INFO  (zkCallback-17-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58b1deeb name:ZooKeeperConnection Watcher:127.0.0.1:55770/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 46785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 46785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 46786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 46790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 46795 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 46798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 46802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 46803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 46808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 46808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 46812 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 46812 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 46816 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 46817 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 46820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 46820 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 46823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 46824 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 46827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 46828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 46831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 46832 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 46836 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 46836 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 46840 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 46841 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 46845 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 46845 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 46851 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 46852 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 46854 INFO  (zkCallback-18-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b7441b name:ZooKeeperConnection Watcher:127.0.0.1:55770 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 46854 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 46854 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 46855 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 46863 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 46865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 46867 INFO  (zkCallback-19-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78c153e name:ZooKeeperConnection Watcher:127.0.0.1:55770/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 46867 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 46868 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 46868 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 46868 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 46874 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 46874 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 46878 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 46879 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 46885 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 46885 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 46891 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 46891 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 46895 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 46896 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 46900 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 46900 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 46904 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 46904 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 46908 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 46908 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 46916 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 46916 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 46920 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 46920 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 46924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 46924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 47255 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores/collection1
   [junit4]   2> 47257 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 47259 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@50918b3{/,null,AVAILABLE}
   [junit4]   2> 47260 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@23745f20{HTTP/1.1}{127.0.0.1:37570}
   [junit4]   2> 47260 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.Server Started @50485ms
   [junit4]   2> 47261 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:35889/hdfs__localhost_35889__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores, hostContext=/, hostPort=37570}
   [junit4]   2> 47261 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 47261 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/'
   [junit4]   2> 47277 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 47278 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 47281 INFO  (zkCallback-20-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7edbe10b name:ZooKeeperConnection Watcher:127.0.0.1:55770/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 47281 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 47282 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 47284 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 47284 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/solr.xml
   [junit4]   2> 47300 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores
   [junit4]   2> 47300 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoreContainer New CoreContainer 320908639
   [junit4]   2> 47300 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/]
   [junit4]   2> 47300 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/lib
   [junit4]   2> 47301 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/lib).
   [junit4]   2> 47309 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 47311 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 47312 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 47312 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 47312 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55770/solr
   [junit4]   2> 47327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 47327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 47329 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 47330 INFO  (zkCallback-22-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@c254735 name:ZooKeeperConnection Watcher:127.0.0.1:55770 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 47331 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 47331 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 47340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 47341 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4bffec54 name:ZooKeeperConnection Watcher:127.0.0.1:55770/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 47341 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 47351 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 47358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 47363 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 47367 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 47376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 47388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 47392 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 47394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 47398 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 47400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37570_
   [junit4]   2> 47403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37570_
   [junit4]   2> 47408 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 47411 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 47413 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 47415 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94304196799627270-127.0.0.1:37570_-n_0000000000
   [junit4]   2> 47420 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37570_
   [junit4]   2> 47421 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 47423 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.Overseer Overseer (id=94304196799627270-127.0.0.1:37570_-n_0000000000) starting
   [junit4]   2> 47431 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 47456 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 47456 INFO  (OverseerCollectionProcessor-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 47456 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 47464 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 47551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 47551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 47555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores
   [junit4]   2> 47559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, loadOnStartup=true, collection=control_collection, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores/collection1/, schema=schema.xml, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores/collection1, transient=false, name=collection1, shard=, coreNodeName=}
   [junit4]   2> 47559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores/collection1/
   [junit4]   2> 47559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 47566 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 47566 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 47571 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 47571 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [n:127.0.0.1:37570_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 47574 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 47574 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 47577 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:37570_",
   [junit4]   2>   "base_url":"http://127.0.0.1:37570"} current state version: 0
   [junit4]   2> 47588 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:37570_",
   [junit4]   2>   "base_url":"http://127.0.0.1:37570"}
   [junit4]   2> 47589 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 47600 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 47604 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 47609 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 48575 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 48575 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 48576 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 48577 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 48578 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 48579 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores/collection1/'
   [junit4]   2> 48602 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 48613 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 48626 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 48678 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 48680 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 48700 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 48968 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 48978 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 48980 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 49001 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 49005 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 49009 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 49010 WARN  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 49010 WARN  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 49011 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 49012 WARN  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 49012 WARN  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 49014 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 49014 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 49014 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 49024 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:35889/solr_hdfs_home
   [junit4]   2> 49024 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 49024 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 49025 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7350eaef
   [junit4]   2> 49046 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 49105 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 49105 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 49112 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 49140 WARN  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:35889/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 49141 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 49159 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 49159 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 49160 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7b3d4f71
   [junit4]   2> 49464 INFO  (IPC Server handler 1 on 35889) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:61000 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-4a2d38ca-be3a-475a-a6fe-9f190c803e39:NORMAL:127.0.0.1:61000|RBW], ReplicaUnderConstruction[[DISK]DS-1238ae68-f13c-45ef-8fad-45ae74939237:NORMAL:127.0.0.1:57356|RBW]]} size 71
   [junit4]   2> 49466 INFO  (IPC Server handler 2 on 35889) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57356 is added to blk_1073741825_1001 size 71
   [junit4]   2> 49888 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b654a183 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6a9d8d3f; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 49888 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 49903 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:35889/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 49907 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 49917 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 49918 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 49918 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 49918 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 49918 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 49919 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 49919 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 49919 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 49922 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 49929 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 49931 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 49933 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 49944 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/get,/admin/ping,/admin/luke,/schema,/admin/segments,/admin/properties,/admin/logging,/admin/system,/update/csv,/config,/replication,standard,/admin/file,/update/json/docs,/admin/threads,/admin/plugins,/update/json,/admin/mbeans
   [junit4]   2> 49946 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 49949 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 49950 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 49968 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 49968 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 49972 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 50050 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b654a183 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6a9d8d3f; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 50051 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 50051 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7fdf6743[collection1] main
   [junit4]   2> 50051 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 50053 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 50053 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 50056 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 50057 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 50057 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 50058 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 50058 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 50058 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 50058 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 50063 INFO  (searcherExecutor-419-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7fdf6743[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 50070 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 50070 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 50070 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 50070 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1508867152421060608
   [junit4]   2> 50070 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0 ms to seed version buckets with highest version 1508867152421060608
   [junit4]   2> 50072 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 50074 INFO  (coreLoadExecutor-418-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 50075 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:37570 collection:control_collection shard:shard1
   [junit4]   2> 50083 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 50095 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94304196799627270-core_node1-n_0000000000
   [junit4]   2> 50098 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 50101 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 50102 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 50105 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 50105 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 50106 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37570/collection1/
   [junit4]   2> 50106 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 50106 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37570/collection1/ has no replicas
   [junit4]   2> 50106 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37570/collection1/ shard1
   [junit4]   2> 50110 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 50116 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 50121 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 50125 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37570",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 50128 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 50130 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 50134 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 50175 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:37570/collection1/ and leader is http://127.0.0.1:37570/collection1/
   [junit4]   2> 50175 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37570
   [junit4]   2> 50176 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 50176 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 50176 INFO  (coreZkRegister-412-thread-1-processing-x:collection1 s:shard1 c:control_collection n:127.0.0.1:37570_ r:core_node1) [n:127.0.0.1:37570_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 50178 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 50180 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:37570_",
   [junit4]   2>   "base_url":"http://127.0.0.1:37570"} current state version: 3
   [junit4]   2> 50181 INFO  (OverseerStateUpdate-94304196799627270-127.0.0.1:37570_-n_0000000000) [n:127.0.0.1:37570_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:37570_",
   [junit4]   2>   "base_url":"http://127.0.0.1:37570"}
   [junit4]   2> 50286 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 50290 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:37570_) [n:127.0.0.1:37570_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 50590 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 50590 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 50592 INFO  (zkCallback-25-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a254ae6 name:ZooKeeperConnection Watcher:127.0.0.1:55770/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 50593 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 50593 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 50593 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 50600 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 50951 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/cores/collection1
   [junit4]   2> 50952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001
   [junit4]   2> 50953 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 50956 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@41cdae7a{/,null,AVAILABLE}
   [junit4]   2> 50957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@eccbcd6{HTTP/1.1}{127.0.0.1:43433}
   [junit4]   2> 50957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.e.j.s.Server Started @54182ms
   [junit4]   2> 50957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=43433, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:35889/hdfs__localhost_35889__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001_tempDir-002_jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 50958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 50958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/'
   [junit4]   2> 50974 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 50975 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 50977 INFO  (zkCallback-26-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@747857de name:ZooKeeperConnection Watcher:127.0.0.1:55770/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 50977 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 50978 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 50980 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 50981 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/solr.xml
   [junit4]   2> 50996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/cores
   [junit4]   2> 50996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoreContainer New CoreContainer 513101830
   [junit4]   2> 50996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/]
   [junit4]   2> 50996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/lib
   [junit4]   2> 50997 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001/shard-1-001/lib).
   [junit4]   2> 51006 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 51007 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 51008 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 51008 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 51008 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55770/solr
   [junit4]   2> 51008 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 51009 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 51010 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[326DF0DCFDA5D52F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 51011 INFO  (zkCallback-28-thread-1) [    ] 

[...truncated too long message...]

]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
   [junit4]    > 	... 11 moreThrowable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=10836, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43825/bpku, http://127.0.0.1:52035/bpku, http://127.0.0.1:42102/bpku, http://127.0.0.1:52097/bpku, http://127.0.0.1:42445/bpku]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([326DF0DCFDA5D52F]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:897)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43825/bpku, http://127.0.0.1:52035/bpku, http://127.0.0.1:42102/bpku, http://127.0.0.1:52097/bpku, http://127.0.0.1:42445/bpku]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:42445/bpku
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:570)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	... 7 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
   [junit4]    > 	... 11 more
   [junit4]   2> 2882692 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[326DF0DCFDA5D52F]-worker) [n:127.0.0.1:52097_bpku c:awholynewstresscollection_collection1_0 s:shard1 r:core_node16 x:awholynewstresscollection_collection1_0_shard1_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_326DF0DCFDA5D52F-001
   [junit4]   2> Aug 07, 2015 2:04:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=4921, maxDocsPerChunk=3, blockSize=1), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=4921, blockSize=1)), sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ja, timezone=Europe/Mariehamn
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=43059720,total=530055168
   [junit4]   2> NOTE: All tests run in this JVM: [StatsComponentTest, XmlUpdateRequestHandlerTest, TestDocSet, OverseerStatusTest, ResponseHeaderTest, TestSystemIdResolver, HdfsBasicDistributedZk2Test, TestStressReorder, QueryEqualityTest, DirectSolrSpellCheckerTest, TestRequestStatusCollectionAPI, TestFieldTypeCollectionResource, TestReplicaProperties, BasicDistributedZk2Test, CachingDirectoryFactoryTest, BufferStoreTest, HLLSerializationTest, TestManagedStopFilterFactory, TestMaxScoreQueryParser, CircularListTest, RecoveryZkTest, TestDefaultSimilarityFactory, TestExpandComponent, TestTrackingShardHandlerFactory, TestBinaryResponseWriter, TestAddFieldRealTimeGet, RuleEngineTest, RulesTest, SpellCheckCollatorTest, HighlighterMaxOffsetTest, VersionInfoTest, TestXIncludeConfig, FullSolrCloudDistribCmdsTest, UnloadDistributedZkTest, HttpPartitionTest, TestSchemaManager, BinaryUpdateRequestHandlerTest, TestStressRecovery, LoggingHandlerTest, TestReloadAndDeleteDocs, TestOrdValues, SampleTest, TestRecovery, TestClusterStateMutator, SparseHLLTest, TestCSVResponseWriter, TestQuerySenderNoQuery, CollectionTooManyReplicasTest, DistribCursorPagingTest, TestBM25SimilarityFactory, TestFreeTextSuggestions, FacetPivotSmallTest, HardAutoCommitTest, PolyFieldTest, SolrIndexSplitterTest, ExplicitHLLTest, TestSchemaNameResource, OutputWriterTest, AutoCommitTest, ResponseLogComponentTest, TestDefaultStatsCache, TestPartialUpdateDeduplication, TestUtils, TestGroupingSearch, PathHierarchyTokenizerFactoryTest, DistributedFacetPivotLargeTest, ChangedSchemaMergeTest, DocValuesTest, SOLR749Test, TestSolrQueryParserResource, TestCopyFieldCollectionResource, TestZkChroot, FileBasedSpellCheckerTest, TestScoreJoinQPNoScore, TestExtendedDismaxParser, TestConfigSetImmutable, TestAuthorizationFramework, BlockDirectoryTest, NotRequiredUniqueKeyTest, TestExactSharedStatsCache, InfoHandlerTest, TestSchemaVersionResource, TestInfoStreamLogging, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [184/531] on J2 in 1007.92s, 1 test, 1 error <<< FAILURES!

[...truncated 1107 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 531 suites, 2104 tests, 2 errors, 26 ignored (14 assumptions)

Total time: 228 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1316998670 bytes
Compression is 0.0%
Took 3 min 51 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 921 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/921/

3 tests failed.
REGRESSION:  org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([85E77981C5A52150:B45CC7B4609A3180]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:765)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:732)
	at org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts(SpellCheckCollatorTest.java:530)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='spellcheck']/lst[@name='collations']/lst[@name='collation']/int[@name='hits' and 6 <= . and . <= 10]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">3</int></lst><result name="response" numFound="0" start="0"></result><lst name="spellcheck"><lst name="suggestions"><lst name="everother"><int name="numFound">1</int><int name="startOffset">9</int><int name="endOffset">18</int><arr name="suggestion"><str>everyother</str></arr></lst></lst><lst name="collations"><lst name="collation"><str name="collationQuery">teststop:everyother</str><int name="hits">12</int><lst name="misspellingsAndCorrections"><str name="everother">everyother</str></lst></lst></lst></lst>
</response>

	request was:spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Aeverother&spellcheck.collateMaxCollectDocs=5
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:758)
	... 41 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at https://127.0.0.1:52289: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:52289: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:301)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:418)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:168)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=1249, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1249, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46144: Could not find collection : awholynewstresscollection_collection1_1
	at __randomizedtesting.SeedInfo.seed([85E77981C5A52150]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10827 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/init-core-data-001
   [junit4]   2> 51135 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 51136 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 51816 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 52656 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 52935 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 52946 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 53035 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 53085 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_48137_hdfs____q0h69n/webapp
   [junit4]   2> 53273 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 54456 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48137
   [junit4]   2> 55418 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 55426 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 55440 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59198_datanode____cggjir/webapp
   [junit4]   2> 55590 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 56066 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59198
   [junit4]   2> 56174 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 56177 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 56195 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_50350_datanode____.2io5ja/webapp
   [junit4]   2> 56317 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 56862 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50350
   [junit4]   2> 56985 INFO  (IPC Server handler 3 on 44015) [    ] BlockStateChange BLOCK* processReport: from storage DS-b5bef40a-5628-4797-a598-97e9477dac0a node DatanodeRegistration(127.0.0.1, datanodeUuid=18bebcc7-9e19-4db8-820d-4d24c0e3764c, infoPort=59198, ipcPort=60699, storageInfo=lv=-56;cid=testClusterID;nsid=799152002;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 56988 INFO  (IPC Server handler 3 on 44015) [    ] BlockStateChange BLOCK* processReport: from storage DS-50de895b-39dc-470f-b494-ee2ee897d468 node DatanodeRegistration(127.0.0.1, datanodeUuid=18bebcc7-9e19-4db8-820d-4d24c0e3764c, infoPort=59198, ipcPort=60699, storageInfo=lv=-56;cid=testClusterID;nsid=799152002;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 56998 INFO  (IPC Server handler 7 on 44015) [    ] BlockStateChange BLOCK* processReport: from storage DS-5f17fad0-fad4-4d2b-b23c-c4a56cd0b95f node DatanodeRegistration(127.0.0.1, datanodeUuid=b89f090e-c73c-4e33-ac48-a98e7489b071, infoPort=50350, ipcPort=37427, storageInfo=lv=-56;cid=testClusterID;nsid=799152002;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 57002 INFO  (IPC Server handler 7 on 44015) [    ] BlockStateChange BLOCK* processReport: from storage DS-c352bcda-90db-489e-aef3-95765e21a1ab node DatanodeRegistration(127.0.0.1, datanodeUuid=b89f090e-c73c-4e33-ac48-a98e7489b071, infoPort=50350, ipcPort=37427, storageInfo=lv=-56;cid=testClusterID;nsid=799152002;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 57759 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 57761 INFO  (Thread-197) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 57762 INFO  (Thread-197) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 57861 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.ZkTestServer start zk server on port:53444
   [junit4]   2> 57878 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 57921 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 57956 INFO  (zkCallback-12-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12b540c9 name:ZooKeeperConnection Watcher:127.0.0.1:53444 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 57956 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 57957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 57959 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 57985 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x14f04103b7e0000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 57988 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 57989 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 57993 INFO  (zkCallback-13-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c70c8e6 name:ZooKeeperConnection Watcher:127.0.0.1:53444/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 57993 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 57994 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 57994 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 57999 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 58004 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 58007 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 58011 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 58011 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 58016 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 58016 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 58021 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 58021 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 58025 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 58025 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 58028 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 58028 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 58031 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 58032 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 58035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 58035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 58038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 58038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 58041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 58042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 58047 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 58047 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 58050 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 58050 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 58370 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores/collection1
   [junit4]   2> 58372 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 58374 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@300059d1{/,null,AVAILABLE}
   [junit4]   2> 58375 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6708c0f5{HTTP/1.1}{127.0.0.1:34206}
   [junit4]   2> 58376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.Server Started @61575ms
   [junit4]   2> 58376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores, hostContext=/, solr.data.dir=hdfs://localhost:44015/hdfs__localhost_44015__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001_tempDir-002_control_data, hostPort=34206}
   [junit4]   2> 58377 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 58377 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/'
   [junit4]   2> 58394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 58394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 58396 INFO  (zkCallback-14-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a05916e name:ZooKeeperConnection Watcher:127.0.0.1:53444/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 58397 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 58397 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 58400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 58400 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/solr.xml
   [junit4]   2> 58417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores
   [junit4]   2> 58417 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoreContainer New CoreContainer 119139155
   [junit4]   2> 58418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/]
   [junit4]   2> 58418 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/lib
   [junit4]   2> 58418 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/lib).
   [junit4]   2> 58427 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 58429 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 58429 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 58430 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 58430 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53444/solr
   [junit4]   2> 58442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 58442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 58443 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 58444 INFO  (zkCallback-16-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b6aa24d name:ZooKeeperConnection Watcher:127.0.0.1:53444 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 58445 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 58446 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 58451 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 58453 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@480a20e3 name:ZooKeeperConnection Watcher:127.0.0.1:53444/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 58453 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 58461 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 58467 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 58471 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 58476 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 58479 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 58489 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 58492 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 58494 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 58496 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34206_
   [junit4]   2> 58498 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34206_
   [junit4]   2> 58502 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 58504 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 58507 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 58508 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94298584962170884-127.0.0.1:34206_-n_0000000000
   [junit4]   2> 58513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34206_
   [junit4]   2> 58514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 58516 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.Overseer Overseer (id=94298584962170884-127.0.0.1:34206_-n_0000000000) starting
   [junit4]   2> 58523 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 58546 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 58547 INFO  (OverseerCollectionProcessor-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 58547 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 58553 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 58670 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 58670 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 58672 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores
   [junit4]   2> 58676 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, collection=control_collection, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores/collection1, coreNodeName=, shard=, name=collection1, configSetProperties=configsetprops.json, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores/collection1/, loadOnStartup=true, dataDir=data/}
   [junit4]   2> 58676 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores/collection1/
   [junit4]   2> 58677 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 58679 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 58679 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 58679 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 58680 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:34206_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 58685 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 58685 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 58687 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:34206_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34206",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 58699 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:34206_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34206",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null}
   [junit4]   2> 58700 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 58725 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 58732 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 58736 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 59686 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 59686 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 59688 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 59688 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 59691 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 59691 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores/collection1/'
   [junit4]   2> 59721 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 59735 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 59753 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 59792 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 59794 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 59820 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 60149 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 60162 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 60165 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 60195 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 60202 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 60207 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 60209 WARN  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 60209 WARN  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 60209 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 60211 WARN  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 60211 WARN  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 60213 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 60214 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 60214 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 60225 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44015/solr_hdfs_home
   [junit4]   2> 60225 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 60225 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 60226 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@607edb0f
   [junit4]   2> 60232 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 60299 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 60300 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 60307 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 60345 WARN  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:44015/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 60346 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 60369 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 60369 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 60370 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=62.203125, floorSegmentMB=2.072265625, forceMergeDeletesPctAllowed=17.10056122100332, segmentsPerTier=17.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4306782061131623
   [junit4]   2> 60722 INFO  (IPC Server handler 6 on 44015) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40639 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-c352bcda-90db-489e-aef3-95765e21a1ab:NORMAL:127.0.0.1:40639|RBW], ReplicaUnderConstruction[[DISK]DS-b5bef40a-5628-4797-a598-97e9477dac0a:NORMAL:127.0.0.1:56496|RBW]]} size 71
   [junit4]   2> 60726 INFO  (IPC Server handler 5 on 44015) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56496 is added to blk_1073741825_1001 size 71
   [junit4]   2> 61124 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7464b09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@64379207; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 61125 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 61141 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:44015/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 61152 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 61153 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 61153 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 61154 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 61154 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 61154 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 61155 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 61156 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 61156 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 61159 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 61163 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 61166 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 61168 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 61188 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/schema,/admin/properties,/update/json/docs,/get,/update/csv,standard,/admin/luke,/admin/segments,/replication,/admin/file,/admin/logging,/admin/ping,/admin/system,/admin/threads,/config,/update,/admin/plugins,/update/json
   [junit4]   2> 61191 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 61193 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 61194 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 61211 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 61212 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 61215 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=48.947265625, floorSegmentMB=1.5087890625, forceMergeDeletesPctAllowed=6.804392023339894, segmentsPerTier=36.0, maxCFSSegmentSizeMB=0.90234375, noCFSRatio=0.8006232487705038
   [junit4]   2> 61286 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b7464b09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@64379207; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 61286 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 61286 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@25e48823[collection1] main
   [junit4]   2> 61287 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 61289 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 61289 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 61292 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 61292 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 61293 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 61294 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 61294 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 61294 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 61294 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 61299 INFO  (searcherExecutor-96-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@25e48823[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 61306 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 61306 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 61306 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 61306 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1508777363027001344
   [junit4]   2> 61307 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1 ms to seed version buckets with highest version 1508777363027001344
   [junit4]   2> 61308 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 61312 INFO  (coreLoadExecutor-95-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 61313 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34206 collection:control_collection shard:shard1
   [junit4]   2> 61321 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 61328 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94298584962170884-core_node1-n_0000000000
   [junit4]   2> 61330 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 61332 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 61333 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 61333 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 61334 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 61334 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34206/collection1/
   [junit4]   2> 61335 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 61335 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34206/collection1/ has no replicas
   [junit4]   2> 61335 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34206/collection1/ shard1
   [junit4]   2> 61337 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 61338 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 61341 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 61346 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 61347 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:34206",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 61452 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 61460 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 61498 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:34206/collection1/ and leader is http://127.0.0.1:34206/collection1/
   [junit4]   2> 61498 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34206
   [junit4]   2> 61498 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 61498 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 61499 INFO  (coreZkRegister-89-thread-1-processing-n:127.0.0.1:34206_ x:collection1 r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:34206_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 61514 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 61515 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:34206_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34206",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 3
   [junit4]   2> 61516 INFO  (OverseerStateUpdate-94298584962170884-127.0.0.1:34206_-n_0000000000) [n:127.0.0.1:34206_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:34206_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:34206",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 61621 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 61624 INFO  (zkCallback-17-thread-1-processing-n:127.0.0.1:34206_) [n:127.0.0.1:34206_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 61691 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 61692 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 61694 INFO  (zkCallback-19-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@384d9a2b name:ZooKeeperConnection Watcher:127.0.0.1:53444/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 61694 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 61695 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 61695 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 61701 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 62027 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/cores/collection1
   [junit4]   2> 62028 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001
   [junit4]   2> 62029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 62041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5438c6fc{/,null,AVAILABLE}
   [junit4]   2> 62042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5b8ff596{HTTP/1.1}{127.0.0.1:39184}
   [junit4]   2> 62042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.e.j.s.Server Started @65241ms
   [junit4]   2> 62042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:44015/hdfs__localhost_44015__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001_tempDir-002_jetty1, hostPort=39184, solrconfig=solrconfig.xml, hostContext=/}
   [junit4]   2> 62043 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 62043 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/'
   [junit4]   2> 62060 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 62062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 62063 INFO  (zkCallback-20-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c1f26f4 name:ZooKeeperConnection Watcher:127.0.0.1:53444/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 62063 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 62064 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 62066 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 62066 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/solr.xml
   [junit4]   2> 62084 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/cores
   [junit4]   2> 62085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoreContainer New CoreContainer 1067189784
   [junit4]   2> 62085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/]
   [junit4]   2> 62085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/lib
   [junit4]   2> 62085 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_85E77981C5A52150-001/shard-1-001/lib).
   [junit4]   2> 62095 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 62096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 62097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 62097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 62097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53444/solr
   [junit4]   2> 62098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 62098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 62099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 62100 INFO  (zkCallback-22-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f77cb6f name:ZooKeeperConnection Watcher:127.0.0.1:53444 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 62101 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 62102 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 62105 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[85E77981C5A52150]) [n:127.0.0.1:39184_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 6

[...truncated too long message...]

Exception: No live SolrServers available to handle this request:[https://127.0.0.1:60634, https://127.0.0.1:34821, https://127.0.0.1:52289, https://127.0.0.1:45018, https://127.0.0.1:59455]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:59455
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:570)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
   [junit4]    > 	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
   [junit4]    > 	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
   [junit4]    > 	... 9 more
   [junit4]   2> 3996559 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[85E77981C5A52150]-worker) [n:127.0.0.1:45018_ c:awholynewcollection_0 s:shard3 r:core_node11 x:awholynewcollection_0_shard3_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_85E77981C5A52150-001
   [junit4]   2> Aug 06, 2015 2:35:55 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {_version_=PostingsFormat(name=Memory doPackFST= false), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), timestamp=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=Memory doPackFST= false), a_i1=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=Memory doPackFST= true), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), range_facet_l_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=et_EE, timezone=Pacific/Norfolk
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=62362712,total=469762048
   [junit4]   2> NOTE: All tests run in this JVM: [SparseHLLTest, LukeRequestHandlerTest, TestFileDictionaryLookup, SpatialFilterTest, DocumentAnalysisRequestHandlerTest, AtomicUpdatesTest, TestRTimer, TestReplicationHandlerBackup, TestBlobHandler, CollectionStateFormat2Test, TestSchemaVersionResource, TestMiniSolrCloudCluster, TestAuthenticationFramework, DataDrivenBlockJoinTest, ScriptEngineTest, ShardRoutingCustomTest, TestStressVersions, TestWriterPerf, TestSuggestSpellingConverter, DeleteInactiveReplicaTest, AlternateDirectoryTest, TestConfigSetImmutable, TestFuzzyAnalyzedSuggestions, TestJettySolrRunner, TestSolrQueryParserDefaultOperatorResource, TestDistributedGrouping, DistributedMLTComponentTest, DocValuesMissingTest, PreAnalyzedFieldTest, SystemInfoHandlerTest, TestLMJelinekMercerSimilarityFactory, TestConfigSets, JSONWriterTest, CachingDirectoryFactoryTest, SimpleMLTQParserTest, TestExpandComponent, AnalysisAfterCoreReloadTest, StatelessScriptUpdateProcessorFactoryTest, TestAuthorizationFramework, TestConfigReload, DistributedQueueTest, AnalyticsMergeStrategyTest, ClusterStateUpdateTest, TestChildDocTransformer, SolrPluginUtilsTest, DeleteLastCustomShardedReplicaTest, TestCoreContainer, TestShardHandlerFactory, UpdateParamsTest, DocValuesMultiTest, TestComponentsName, ZkControllerTest, TestCustomSort, BadComponentTest, AsyncMigrateRouteKeyTest, TestCloudManagedSchemaConcurrent, DistanceFunctionTest, TestAnalyzedSuggestions, TestIndexSearcher, TestMacros, ResourceLoaderTest, TestSearchPerf, TestLeaderElectionZkExpiry, TestAtomicUpdateErrorCases, CopyFieldTest, ShardRoutingTest, AliasIntegrationTest, SolrCloudExampleTest, TestFreeTextSuggestions, FieldMutatingUpdateProcessorTest, OutOfBoxZkACLAndCredentialsProvidersTest, DisMaxRequestHandlerTest, DistributedTermsComponentTest, TestSortByMinMaxFunction, VMParamsZkACLAndCredentialsProvidersTest, HighlighterMaxOffsetTest, SliceStateTest, TestManagedStopFilterFactory, MergeStrategyTest, SimpleCollectionCreateDeleteTest, TestLRUStatsCache, AssignTest, TestLMDirichletSimilarityFactory, ZkSolrClientTest, OpenCloseCoreStressTest, SpellingQueryConverterTest, ZkStateWriterTest, TestCharFilters, AnalyticsQueryTest, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [407/527] on J1 in 1926.20s, 1 test, 1 error <<< FAILURES!

[...truncated 380 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 527 suites, 2092 tests, 3 errors, 26 ignored (14 assumptions)

Total time: 206 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1303829980 bytes
Compression is 0.0%
Took 3 min 36 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 920 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/920/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
KeeperErrorCode = Session expired for /live_nodes

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:328)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:325)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:325)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:487)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:225)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoCollectionSpecified(CollectionsAPIDistributedZkTest.java:464)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15246, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15246, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:60089: Could not find collection : awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([599966DD976D72CE]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10648 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/init-core-data-001
   [junit4]   2> 127015 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[599966DD976D72CE]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 127016 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[599966DD976D72CE]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /wmya/k
   [junit4]   2> 127020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 127021 INFO  (Thread-1069) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 127021 INFO  (Thread-1069) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 127121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.ZkTestServer start zk server on port:60384
   [junit4]   2> 127121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 127122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 127125 INFO  (zkCallback-55-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@551b91c2 name:ZooKeeperConnection Watcher:127.0.0.1:60384 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 127126 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 127126 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 127129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 127131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 127132 INFO  (zkCallback-56-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8935f8f name:ZooKeeperConnection Watcher:127.0.0.1:60384/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 127132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 127132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 127135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 127137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 127138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 127140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 127140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 127143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 127143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 127145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 127145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 127146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 127147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 127148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 127149 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 127151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 127151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 127152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 127153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 127154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 127155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 127156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 127157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 127159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 127159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 127160 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 127161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 127163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 127164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 127165 INFO  (zkCallback-57-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2fddd93e name:ZooKeeperConnection Watcher:127.0.0.1:60384 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127165 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 127166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 127166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 127169 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 127169 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 127170 INFO  (zkCallback-58-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b550974 name:ZooKeeperConnection Watcher:127.0.0.1:60384/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127171 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 127171 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 127172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 127172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 127175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 127176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 127178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 127178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 127179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 127180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 127181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 127182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 127183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 127183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 127185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 127185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 127187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 127187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 127189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 127189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 127191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 127191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 127192 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 127193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 127194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 127195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 127490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1
   [junit4]   2> 127492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 127495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5fb571a1{/wmya/k,null,AVAILABLE}
   [junit4]   2> 127498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4ebb5031{HTTP/1.1}{127.0.0.1:37654}
   [junit4]   2> 127498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.Server Started @130437ms
   [junit4]   2> 127499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/tempDir-001/control/data, hostContext=/wmya/k, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores, hostPort=37654}
   [junit4]   2> 127499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 127499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/'
   [junit4]   2> 127515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 127516 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 127518 INFO  (zkCallback-59-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2841634b name:ZooKeeperConnection Watcher:127.0.0.1:60384/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 127519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 127521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 127521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/solr.xml
   [junit4]   2> 127535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores
   [junit4]   2> 127536 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoreContainer New CoreContainer 720900901
   [junit4]   2> 127536 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/]
   [junit4]   2> 127536 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/lib
   [junit4]   2> 127536 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/lib).
   [junit4]   2> 127545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 127546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 127547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 127547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 127547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60384/solr
   [junit4]   2> 127547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 127547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 127548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 127550 INFO  (zkCallback-61-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@23cb08f3 name:ZooKeeperConnection Watcher:127.0.0.1:60384 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 127551 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 127554 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 127556 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@355ab50a name:ZooKeeperConnection Watcher:127.0.0.1:60384/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127556 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 127558 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 127560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 127563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 127565 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 127567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 127569 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 127570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 127572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 127573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37654_wmya%2Fk
   [junit4]   2> 127573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37654_wmya%2Fk
   [junit4]   2> 127575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 127576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 127577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 127578 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000
   [junit4]   2> 127579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37654_wmya%2Fk
   [junit4]   2> 127579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 127580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.Overseer Overseer (id=94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) starting
   [junit4]   2> 127582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 127588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 127588 INFO  (OverseerCollectionProcessor-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 127588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 127589 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 127623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 127624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 127624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores
   [junit4]   2> 127625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, name=collection1, dataDir=data/, collection=control_collection, config=solrconfig.xml, loadOnStartup=true, coreNodeName=, schema=schema.xml, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1, shard=}
   [junit4]   2> 127625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/
   [junit4]   2> 127626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 127627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 127627 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 127628 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 127628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 127629 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 127629 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 127630 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:37654_wmya%2Fk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37654/wmya/k",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 127631 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:37654_wmya%2Fk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37654/wmya/k",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 127631 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 127631 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 127633 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 127634 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 128629 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 128629 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 128630 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 128630 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 128631 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 128631 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/'
   [junit4]   2> 128649 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 128658 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 128673 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 128706 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 128708 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 128716 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 129006 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 129016 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 129018 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 129031 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 129034 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 129037 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 129038 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 129038 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 129038 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 129039 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 129039 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 129040 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 129040 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 129040 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 129041 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 129041 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2554136a
   [junit4]   2> 129042 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/data
   [junit4]   2> 129042 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/data/index/
   [junit4]   2> 129042 WARN  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 129043 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/data/index
   [junit4]   2> 129043 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6893608296025293]
   [junit4]   2> 129045 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@23fd06cc),segFN=segments_1,generation=1}
   [junit4]   2> 129045 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 129046 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/data/
   [junit4]   2> 129046 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 129051 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 129052 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 129052 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 129052 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 129052 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 129052 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 129053 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 129053 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 129053 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 129056 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 129057 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 129059 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 129060 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 129064 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /get,/admin/file,/update/json,/admin/luke,/admin/logging,standard,/update,/replication,/config,/schema,/admin/properties,/update/json/docs,/admin/segments,/admin/mbeans,/admin/threads,/admin/system,/update/csv,/admin/plugins,/admin/ping
   [junit4]   2> 129066 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 129066 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 129067 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 129067 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 129068 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 129068 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=77.6630859375, floorSegmentMB=1.5595703125, forceMergeDeletesPctAllowed=26.30914737620219, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8009856768712821
   [junit4]   2> 129069 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@23fd06cc),segFN=segments_1,generation=1}
   [junit4]   2> 129070 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 129070 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@22070cee[collection1] main
   [junit4]   2> 129070 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 129071 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 129071 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 129071 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 129072 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 129072 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 129072 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 129073 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 129073 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 129073 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 129074 INFO  (searcherExecutor-240-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@22070cee[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 129074 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 129074 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 129074 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 129074 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1508698742486728704
   [junit4]   2> 129076 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1508698742486728704
   [junit4]   2> 129077 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 129077 INFO  (coreLoadExecutor-239-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 129078 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:37654/wmya/k collection:control_collection shard:shard1
   [junit4]   2> 129079 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 129082 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94293671270940678-core_node1-n_0000000000
   [junit4]   2> 129083 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 129085 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 129085 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 129085 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 129085 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37654/wmya/k/collection1/
   [junit4]   2> 129085 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 129085 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 129085 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37654/wmya/k/collection1/ has no replicas
   [junit4]   2> 129086 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37654/wmya/k/collection1/ shard1
   [junit4]   2> 129086 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 129093 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 129094 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:37654/wmya/k",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 129130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 129130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 129132 INFO  (zkCallback-64-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22b258d2 name:ZooKeeperConnection Watcher:127.0.0.1:60384/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 129132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 129132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 129133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 129136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 129196 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 129196 INFO  (zkCallback-64-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 129199 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 129199 INFO  (zkCallback-64-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 129242 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:37654/wmya/k/collection1/ and leader is http://127.0.0.1:37654/wmya/k/collection1/
   [junit4]   2> 129242 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37654/wmya/k
   [junit4]   2> 129242 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 129242 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 129243 INFO  (coreZkRegister-233-thread-1-processing-c:control_collection r:core_node1 s:shard1 n:127.0.0.1:37654_wmya%2Fk x:collection1) [n:127.0.0.1:37654_wmya%2Fk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 129244 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 129246 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:37654_wmya%2Fk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37654/wmya/k",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 129246 INFO  (OverseerStateUpdate-94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:37654_wmya%2Fk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:37654/wmya/k",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 129349 INFO  (zkCallback-64-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 129349 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 129351 INFO  (zkCallback-64-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 129352 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 129432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/cores/collection1
   [junit4]   2> 129433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001
   [junit4]   2> 129434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 129436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6e68aee9{/wmya/k,null,AVAILABLE}
   [junit4]   2> 129437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7d795311{HTTP/1.1}{127.0.0.1:48883}
   [junit4]   2> 129437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.e.j.s.Server Started @132376ms
   [junit4]   2> 129437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/cores, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/tempDir-001/jetty1, hostContext=/wmya/k, hostPort=48883}
   [junit4]   2> 129438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 129439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/'
   [junit4]   2> 129456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 129456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 129458 INFO  (zkCallback-65-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3455f9e6 name:ZooKeeperConnection Watcher:127.0.0.1:60384/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 129458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 129459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 129460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 129461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/solr.xml
   [junit4]   2> 129475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/cores
   [junit4]   2> 129475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoreContainer New CoreContainer 1291330084
   [junit4]   2> 129475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/]
   [junit4]   2> 129475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/lib
   [junit4]   2> 129475 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/lib).
   [junit4]   2> 129483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 129485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 129485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 129485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 129485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60384/solr
   [junit4]   2> 129486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 129486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 129486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 129488 INFO  (zkCallback-67-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73f1e01b name:ZooKeeperConnection Watcher:127.0.0.1:60384 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 129489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 129489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 129491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 129492 INFO  (zkCallback-68-thread-1-processing-n:127.0.0.1:48883_wmya%2Fk) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32ff3c9e name:ZooKeeperConnection Watcher:127.0.0.1:60384/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 129493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 129498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 130503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48883_wmya%2Fk
   [junit4]   2> 130505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48883_wmya%2Fk
   [junit4]   2> 130510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 130511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94293671270940682-127.0.0.1:48883_wmya%2Fk-n_0000000001
   [junit4]   2> 130512 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94293671270940678-127.0.0.1:37654_wmya%2Fk-n_0000000000 to know if I could be the leader
   [junit4]   2> 130538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 130539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 130540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/cores
   [junit4]   2> 130542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, collection=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/cores/collection1, schema=schema.xml, name=collection1, coreNodeName=, config=solrconfig.xml, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/cores/collection1/, dataDir=data/, transient=false, loadOnStartup=true}
   [junit4]   2> 130542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_599966DD976D72CE-001/shard-1-001/cores/collection1/
   [junit4]   2> 130542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 130544 INFO  (coreLoadExecutor-250-thread-1-processing-n:127.0.0.1:48883_wmya%2Fk) [n:127.0.0.1:48883_wmya%2Fk c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 130544 INFO  (coreLoadExecutor-250-thread-1-processing-n:127.0.0.1:48883_wmya%2Fk) [n:127.0.0.1:48883_wmya%2Fk c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 130545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 130545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[599966DD976D72CE]) [n:127.0.0.1:48883_wmya%2Fk    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 130545 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:37654_wmya%2Fk) [n:127.0.0.1:37654_wmya%2Fk    ] o.a.s.c.DistributedQueue NodeChildrenChanged

[...truncated too long message...]

ltMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3920961 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[599966DD976D72CE]-worker) [n:127.0.0.1:35439_ c:addReplicaColl s:shard2 r:core_node1 x:addReplicaColl_shard2_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 05, 2015 5:43:55 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), id=Lucene50(blocksize=128), range_facet_l_dv=Lucene50(blocksize=128), intDefault=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=Asserting), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), text=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=LuceneVarGapFixedInterval), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=et, timezone=America/Porto_Velho
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=93933200,total=525860864
   [junit4]   2> NOTE: All tests run in this JVM: [TestBinaryField, TestSolr4Spatial, TestManagedSchemaFieldTypeResource, TestSolrConfigHandler, TestReRankQParserPlugin, TestReversedWildcardFilterFactory, TestShortCircuitedRequests, QueryElevationComponentTest, TestSweetSpotSimilarityFactory, DeleteShardTest, TestJmxIntegration, TestRemoteStreaming, RecoveryAfterSoftCommitTest, TestManagedSynonymFilterFactory, PingRequestHandlerTest, LeaderElectionIntegrationTest, TestCloudInspectUtil, CustomCollectionTest, TestConfigOverlay, BasicZkTest, TestReqParamsAPI, BasicDistributedZkTest, DistributedFacetPivotSmallAdvancedTest, SchemaVersionSpecificBehaviorTest, CleanupOldIndexTest, UUIDFieldTest, TestTrieFacet, CurrencyFieldXmlFileTest, TestHighlightDedupGrouping, TestWordDelimiterFilterFactory, TestOverriddenPrefixQueryForCustomFieldType, DateFieldTest, AnalysisErrorHandlingTest, TestReplicationHandler, HdfsNNFailoverTest, TestSolrIndexConfig, TestClusterStateMutator, TestManagedResource, TestJoin, TestDocumentBuilder, TestIBSimilarityFactory, TestComplexPhraseQParserPlugin, SuggesterWFSTTest, SolrRequestParserTest, ChaosMonkeyNothingIsSafeTest, PeerSyncTest, LeaderInitiatedRecoveryOnCommitTest, MultiTermTest, DebugComponentTest, BlockCacheTest, TestFaceting, CollectionTooManyReplicasTest, RollingRestartTest, MultiThreadedOCPTest, TestFieldSortValues, TestJmxMonitoredMap, TestHashQParserPlugin, SolrXmlInZkTest, TestSearcherReuse, TestRestManager, CollectionsAPIAsyncDistributedZkTest, SimplePostToolTest, TestSolr4Spatial2, TestIndexingPerformance, TestCSVLoader, TestInfoStreamLogging, SuggestComponentTest, StandardRequestHandlerTest, TestPhraseSuggestions, TestArbitraryIndexDir, SaslZkACLProviderTest, DistributedFacetPivotWhiteBoxTest, TestNRTOpen, DateMathParserTest, TestRandomRequestDistribution, TestFunctionQuery, DefaultValueUpdateProcessorTest, TestRealTimeGet, TestCloudPivotFacet, HighlighterTest, HLLUtilTest, TestFieldCollectionResource, TestBulkSchemaConcurrent, TestPKIAuthenticationPlugin, SolrIndexConfigTest, ReplicationFactorTest, ShardSplitTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [471/527] on J0 in 631.87s, 1 test, 1 error <<< FAILURES!

[...truncated 191 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 527 suites, 2092 tests, 2 errors, 26 ignored (14 assumptions)

Total time: 197 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1382787708 bytes
Compression is 0.0%
Took 3 min 53 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 919 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/919/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=69621, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=69621, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37406/c_msh: Could not find collection : awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([661D7B24D2BB1486]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=813, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=813, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:52030/i_ct/rc: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([661D7B24D2BB1486]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10763 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 59   INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /i_ct/rc
   [junit4]   2> 1338 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2734 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3070 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 3107 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3231 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3290 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_58049_hdfs____.c5kkro/webapp
   [junit4]   2> 3591 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4641 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58049
   [junit4]   2> 5819 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 5829 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 5846 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57738_datanode____g0d2ox/webapp
   [junit4]   2> 5988 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 6343 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57738
   [junit4]   2> 6574 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 6579 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 6606 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_44084_datanode____z4pehf/webapp
   [junit4]   2> 6790 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 7286 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44084
   [junit4]   2> 8104 INFO  (IPC Server handler 6 on 57963) [    ] BlockStateChange BLOCK* processReport: from storage DS-862c39e8-f601-4862-b0c8-71900415e0dc node DatanodeRegistration(127.0.0.1, datanodeUuid=3c911e0d-08c8-41f3-9c6b-ae91bdcd256e, infoPort=57738, ipcPort=47010, storageInfo=lv=-56;cid=testClusterID;nsid=1244829943;c=0), blocks: 0, hasStaleStorages: true, processing time: 4 msecs
   [junit4]   2> 8106 INFO  (IPC Server handler 8 on 57963) [    ] BlockStateChange BLOCK* processReport: from storage DS-178929aa-b44f-40e7-9286-7c87207622d9 node DatanodeRegistration(127.0.0.1, datanodeUuid=87e21a85-cf3f-46b8-927f-ebcda55a7f59, infoPort=44084, ipcPort=47605, storageInfo=lv=-56;cid=testClusterID;nsid=1244829943;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 8107 INFO  (IPC Server handler 6 on 57963) [    ] BlockStateChange BLOCK* processReport: from storage DS-91023023-e5fb-454f-856f-655bf822eec6 node DatanodeRegistration(127.0.0.1, datanodeUuid=3c911e0d-08c8-41f3-9c6b-ae91bdcd256e, infoPort=57738, ipcPort=47010, storageInfo=lv=-56;cid=testClusterID;nsid=1244829943;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 8110 INFO  (IPC Server handler 8 on 57963) [    ] BlockStateChange BLOCK* processReport: from storage DS-775127e4-545b-48b5-a2b0-ed118e8d1fee node DatanodeRegistration(127.0.0.1, datanodeUuid=87e21a85-cf3f-46b8-927f-ebcda55a7f59, infoPort=44084, ipcPort=47605, storageInfo=lv=-56;cid=testClusterID;nsid=1244829943;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 8889 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 8892 INFO  (Thread-111) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 8894 INFO  (Thread-111) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 8992 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.ZkTestServer start zk server on port:56196
   [junit4]   2> 9027 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 9071 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 9118 INFO  (zkCallback-1-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@302569cc name:ZooKeeperConnection Watcher:127.0.0.1:56196 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 9120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 9123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 9126 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 9190 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 9199 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 9200 INFO  (zkCallback-2-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4fa0da4f name:ZooKeeperConnection Watcher:127.0.0.1:56196/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 9201 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 9203 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 9210 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 9220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 9226 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 9230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 9236 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 9251 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 9259 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 9260 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 9269 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 9270 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 9279 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 9280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 9285 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 9286 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 9290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 9291 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 9307 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 9308 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 9312 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 9313 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 9318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 9318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 9322 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 9323 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 9327 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 9328 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 9343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 9344 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 9347 INFO  (zkCallback-3-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10dce3f6 name:ZooKeeperConnection Watcher:127.0.0.1:56196/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 9347 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 9348 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 9354 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 9357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 9364 INFO  (zkCallback-4-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@31de478b name:ZooKeeperConnection Watcher:127.0.0.1:56196 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 9365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 9365 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 9366 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 9374 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 9376 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 9379 INFO  (zkCallback-5-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@707924f2 name:ZooKeeperConnection Watcher:127.0.0.1:56196/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 9380 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 9380 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 9381 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 9382 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 9388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 9388 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 9393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 9393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 9398 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 9398 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 9402 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 9403 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 9407 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 9407 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 9411 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 9412 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 9416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 9416 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 9420 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 9421 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 9425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 9425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 9429 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 9430 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 9435 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 9435 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 9833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores/collection1
   [junit4]   2> 9865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.u.log Logging initialized @13309ms
   [junit4]   2> 10025 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 10097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@28712258{/i_ct/rc,null,AVAILABLE}
   [junit4]   2> 10338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4449c1d0{SSL-http/1.1}{127.0.0.1:52030}
   [junit4]   2> 10339 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.Server Started @13783ms
   [junit4]   2> 10340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/i_ct/rc, hostPort=52030, solr.data.dir=hdfs://localhost:57963/hdfs__localhost_57963__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores}
   [junit4]   2> 10539 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 10564 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/'
   [junit4]   2> 10721 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 10723 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 10734 INFO  (zkCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3becb65 name:ZooKeeperConnection Watcher:127.0.0.1:56196/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 10735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 10736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 10741 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 10744 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/solr.xml
   [junit4]   2> 10879 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores
   [junit4]   2> 10909 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoreContainer New CoreContainer 474712640
   [junit4]   2> 10909 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/]
   [junit4]   2> 10910 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/lib
   [junit4]   2> 10910 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/lib).
   [junit4]   2> 10944 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 10973 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 10980 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 10982 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 10984 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56196/solr
   [junit4]   2> 10998 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 10999 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 11000 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 11003 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3688790a name:ZooKeeperConnection Watcher:127.0.0.1:56196 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 11004 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 11005 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 11015 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 11017 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@419e2223 name:ZooKeeperConnection Watcher:127.0.0.1:56196/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 11018 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 11027 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 11035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 11041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 11048 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 11056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 11073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 11077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 11081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 11083 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52030_i_ct%2Frc
   [junit4]   2> 11086 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52030_i_ct%2Frc
   [junit4]   2> 11091 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 11094 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 11098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 11100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000
   [junit4]   2> 11106 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52030_i_ct%2Frc
   [junit4]   2> 11107 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 11111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.Overseer Overseer (id=94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) starting
   [junit4]   2> 11118 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 11145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 11146 INFO  (OverseerCollectionProcessor-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 11147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 11160 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 11161 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 11168 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 11194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores
   [junit4]   2> 11262 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, transient=false, collection=control_collection, config=solrconfig.xml, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores/collection1/, name=collection1, loadOnStartup=true, configSetProperties=configsetprops.json, dataDir=data/, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores/collection1}
   [junit4]   2> 11264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores/collection1/
   [junit4]   2> 11264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 11269 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 11269 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 11279 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 11279 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 11283 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 11284 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 11286 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:52030/i_ct/rc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:52030_i_ct%2Frc",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 11298 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:52030/i_ct/rc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:52030_i_ct%2Frc",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 11300 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 11310 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 11315 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 11320 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 12284 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 12285 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 12286 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 12286 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 12289 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 12290 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores/collection1/'
   [junit4]   2> 12345 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 12378 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12480 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 12705 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12717 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 12891 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 13747 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 13801 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 13831 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 14076 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 14084 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 14095 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 14098 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 14098 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 14099 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 14101 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 14101 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 14106 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 14107 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 14108 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 14122 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:57963/solr_hdfs_home
   [junit4]   2> 14123 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 14132 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 14136 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3879c593
   [junit4]   2> 14171 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 14306 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 14310 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 14319 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 14422 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:57963/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 14438 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 14489 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 14489 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 14491 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=8, maxMergedSegmentMB=72.59765625, floorSegmentMB=0.5341796875, forceMergeDeletesPctAllowed=2.1478080441866156, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 14874 INFO  (IPC Server handler 2 on 57963) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50000 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-862c39e8-f601-4862-b0c8-71900415e0dc:NORMAL:127.0.0.1:41576|RBW], ReplicaUnderConstruction[[DISK]DS-775127e4-545b-48b5-a2b0-ed118e8d1fee:NORMAL:127.0.0.1:50000|RBW]]} size 71
   [junit4]   2> 14880 INFO  (IPC Server handler 0 on 57963) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41576 is added to blk_1073741825_1001 size 71
   [junit4]   2> 15287 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8e51915 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44ac2f9b; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 15289 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 15293 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:57963/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 15656 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 15678 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 15683 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 15684 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 15685 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 15685 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 15708 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 15709 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 15710 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 15771 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 15779 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 15782 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 15785 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 15820 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/admin/properties,/replication,/admin/segments,/update/json,/admin/file,/admin/mbeans,/get,/schema,/update/csv,/admin/system,/admin/logging,/admin/ping,standard,/admin/luke,/admin/threads,/update/json/docs,/config,/admin/plugins
   [junit4]   2> 15823 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 15845 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 15845 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 15879 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 15879 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 15884 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=32.3017578125, floorSegmentMB=1.2001953125, forceMergeDeletesPctAllowed=23.832312960512834, segmentsPerTier=8.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 15975 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b8e51915 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44ac2f9b; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 15976 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 15999 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4f81f1ac[collection1] main
   [junit4]   2> 16011 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 16014 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 16014 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 16019 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 16019 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 16047 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 16049 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 16049 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 16050 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 16080 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 16087 INFO  (searcherExecutor-8-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4f81f1ac[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 16099 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 16100 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 16100 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 16101 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1508608070680313856
   [junit4]   2> 16101 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1 ms to seed version buckets with highest version 1508608070680313856
   [junit4]   2> 16103 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 16105 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 16107 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:52030/i_ct/rc collection:control_collection shard:shard1
   [junit4]   2> 16115 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 16129 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94288003950247943-core_node1-n_0000000000
   [junit4]   2> 16132 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 16135 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 16138 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 16145 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 16150 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 16150 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 16151 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 16152 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:52030/i_ct/rc/collection1/
   [junit4]   2> 16152 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 16152 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:52030/i_ct/rc/collection1/ has no replicas
   [junit4]   2> 16153 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:52030/i_ct/rc/collection1/ shard1
   [junit4]   2> 16155 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 16167 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 16170 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:52030/i_ct/rc",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 16277 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 16281 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 16318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 16319 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 16320 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:52030/i_ct/rc/collection1/ and leader is https://127.0.0.1:52030/i_ct/rc/collection1/
   [junit4]   2> 16321 INFO  (zkCallback-11-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4299eb1d name:ZooKeeperConnection Watcher:127.0.0.1:56196/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 16322 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52030/i_ct/rc
   [junit4]   2> 16323 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 16322 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 16323 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 16324 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 s:shard1 c:control_collection r:core_node1 n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 16324 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 16324 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 16331 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 16334 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:52030/i_ct/rc",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:52030_i_ct%2Frc",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 16341 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 16342 INFO  (OverseerStateUpdate-94288003950247943-127.0.0.1:52030_i_ct%2Frc-n_0000000000) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:52030/i_ct/rc",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:52030_i_ct%2Frc",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 16453 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 16453 INFO  (zkCallback-11-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 16459 INFO  (zkCallback-11-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 16459 INFO  (zkCallback-9-thread-1-processing-n:127.0.0.1:52030_i_ct%2Frc) [n:127.0.0.1:52030_i_ct%2Frc    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 16793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/cores/collection1
   [junit4]   2> 16795 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001
   [junit4]   2> 16800 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 16825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4e0c85da{/i_ct/rc,null,AVAILABLE}
   [junit4]   2> 16847 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4659c680{SSL-http/1.1}{127.0.0.1:33126}
   [junit4]   2> 16847 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.e.j.s.Server Started @20291ms
   [junit4]   2> 16848 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:57963/hdfs__localhost_57963__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001_tempDir-002_jetty1, hostPort=33126, hostContext=/i_ct/rc, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 16851 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 16851 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/'
   [junit4]   2> 16872 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 16875 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 16876 INFO  (zkCallback-12-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1075dedd name:ZooKeeperConnection Watcher:127.0.0.1:56196/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 16876 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 16877 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 16880 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 16880 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/solr.xml
   [junit4]   2> 16940 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/cores
   [junit4]   2> 16941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoreContainer New CoreContainer 1819628590
   [junit4]   2> 16941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/]
   [junit4]   2> 16942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/lib
   [junit4]   2> 16942 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_661D7B24D2BB1486-001/shard-1-001/lib).
   [junit4]   2> 16961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 16967 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 16970 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 16970 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 16971 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56196/solr
   [junit4]   2> 16971 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[661D7B24D2BB1486]) [   

[...truncated too long message...]

.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=69622, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37406/c_msh: Could not find collection : awholynewstresscollection_collection5_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([661D7B24D2BB1486]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=69618, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37406/c_msh: Could not find collection : awholynewstresscollection_collection1_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([661D7B24D2BB1486]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=69619, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37406/c_msh: Could not find collection : awholynewstresscollection_collection2_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([661D7B24D2BB1486]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)
   [junit4]   2> 2870302 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[661D7B24D2BB1486]-worker) [n:127.0.0.1:52508_c_msh c:addReplicaColl s:shard2 r:core_node2 x:addReplicaColl_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {a_i1=Lucene50(blocksize=128), _version_=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=Asserting), multiDefault=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=Lucene50(blocksize=128), other_tl1=PostingsFormat(name=Asserting), range_facet_i_dv=Lucene50(blocksize=128), text=Lucene50(blocksize=128), a_t=Lucene50(blocksize=128), timestamp=Lucene50(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=en_CA, timezone=ROK
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=147825752,total=493879296
   [junit4]   2> NOTE: All tests run in this JVM: [HdfsCollectionsAPIDistributedZkTest, FileUtilsTest, ShardRoutingCustomTest, RequestLoggingTest, TestElisionMultitermQuery, ChangedSchemaMergeTest, TestLMDirichletSimilarityFactory, CoreAdminCreateDiscoverTest, DisMaxRequestHandlerTest, TestSolrCloudWithKerberosAlt, TestStressVersions, SegmentsInfoRequestHandlerTest, TestFastWriter, TestSchemaResource, CollectionReloadTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestInfoStreamLogging, TestCopyFieldCollectionResource, RankQueryTest, TestValueSourceCache, ShardRoutingTest, HdfsUnloadDistributedZkTest, ZkCLITest, IndexSchemaRuntimeFieldTest, ChaosMonkeySafeLeaderTest, SuggesterTSTTest, TestDefaultStatsCache, HdfsBasicDistributedZkTest, TestFiltering, TestSolrQueryParserResource, TestOmitPositions, HdfsDirectoryTest, TestFuzzyAnalyzedSuggestions, StressHdfsTest, TestComplexPhraseQParserPlugin, TestManagedSynonymFilterFactory, PreAnalyzedFieldTest, RecoveryAfterSoftCommitTest, TestDistributedGrouping, TestSchemaManager, DocExpirationUpdateProcessorFactoryTest, RequiredFieldsTest, UpdateParamsTest, JSONWriterTest, TestWriterPerf, SolrCmdDistributorTest, SpatialHeatmapFacetsTest, TestComponentsName, TestStandardQParsers, TestCharFilters, SolrXmlInZkTest, SpatialFilterTest, DirectSolrConnectionTest, ScriptEngineTest, TestBulkSchemaConcurrent, TestConfigOverlay, BasicFunctionalityTest, TestRestoreCore, DistributedExpandComponentTest, JsonLoaderTest, TestBlendedInfixSuggestions, HdfsSyncSliceTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [359/523] on J1 in 249.67s, 1 test, 1 error <<< FAILURES!

[...truncated 548 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 523 suites, 2084 tests, 2 errors, 25 ignored (14 assumptions)

Total time: 192 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1292435797 bytes
Compression is 0.0%
Took 3 min 28 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 918 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/918/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12871, name=Thread-7299, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12871, name=Thread-7299, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
	at __randomizedtesting.SeedInfo.seed([ECA2FE4A899C4B36:64F6C190276026CE]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:36420/bv/j/collection2_shard2_replica2
	at __randomizedtesting.SeedInfo.seed([ECA2FE4A899C4B36]:0)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:645)
Caused by: org.apache.solr.client.solrj.SolrServerException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:36420/bv/j/collection2_shard2_replica2
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:641)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:967)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:642)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:36420/bv/j/collection2_shard2_replica2
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:635)
	... 6 more
Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:36420 failed to respond
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 11 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=69881, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=69881, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35400: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([ECA2FE4A899C4B36]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 11199 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/init-core-data-001
   [junit4]   2> 1505654 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[ECA2FE4A899C4B36]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /bv/j
   [junit4]   2> 1505672 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1505673 INFO  (Thread-7127) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1505673 INFO  (Thread-7127) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1505773 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.ZkTestServer start zk server on port:55464
   [junit4]   2> 1505773 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1505790 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1505809 INFO  (zkCallback-1040-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4123eb67 name:ZooKeeperConnection Watcher:127.0.0.1:55464 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1505809 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1505809 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1505810 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1505813 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1505829 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1505841 INFO  (zkCallback-1041-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6185a771 name:ZooKeeperConnection Watcher:127.0.0.1:55464/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1505841 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1505842 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1505842 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1505846 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1505848 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1505849 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1505851 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1505852 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1505856 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1505857 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1505858 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1505859 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1505860 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1505860 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1505862 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1505862 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1505872 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1505873 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1505875 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1505875 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1505886 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1505887 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1505892 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1505893 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1505895 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1505895 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1505897 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1505897 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1506352 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1
   [junit4]   2> 1506353 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1506357 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c556444{/bv/j,null,AVAILABLE}
   [junit4]   2> 1506358 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5a29e3ec{HTTP/1.1}{127.0.0.1:57015}
   [junit4]   2> 1506358 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.Server Started @1509729ms
   [junit4]   2> 1506358 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores, hostContext=/bv/j, hostPort=57015}
   [junit4]   2> 1506358 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1506359 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/'
   [junit4]   2> 1506380 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1506381 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1506383 INFO  (zkCallback-1042-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e4f6c62 name:ZooKeeperConnection Watcher:127.0.0.1:55464/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1506383 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1506383 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1506384 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1506385 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/solr.xml
   [junit4]   2> 1506410 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores
   [junit4]   2> 1506410 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoreContainer New CoreContainer 16536487
   [junit4]   2> 1506410 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/]
   [junit4]   2> 1506410 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/lib
   [junit4]   2> 1506411 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/lib).
   [junit4]   2> 1506422 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1506424 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1506424 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1506424 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1506424 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55464/solr
   [junit4]   2> 1506425 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1506425 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1506426 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1506428 INFO  (zkCallback-1044-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54761300 name:ZooKeeperConnection Watcher:127.0.0.1:55464 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1506429 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1506430 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1506433 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1506435 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19694b4f name:ZooKeeperConnection Watcher:127.0.0.1:55464/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1506435 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1506437 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1506439 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1506441 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1506444 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1506447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1506449 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1506451 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1506452 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1506454 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57015_bv%2Fj
   [junit4]   2> 1506455 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57015_bv%2Fj
   [junit4]   2> 1506457 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1506459 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1506460 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1506461 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000
   [junit4]   2> 1506462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57015_bv%2Fj
   [junit4]   2> 1506462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1506463 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer Overseer (id=94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) starting
   [junit4]   2> 1506467 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1506475 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1506476 INFO  (OverseerCollectionProcessor-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 1506477 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1506478 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1506485 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1506487 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1506488 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores
   [junit4]   2> 1506489 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, name=collection1, config=solrconfig.xml, collection=control_collection, schema=schema.xml, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1, dataDir=data/, coreNodeName=, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/, loadOnStartup=true}
   [junit4]   2> 1506489 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/
   [junit4]   2> 1506490 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1506491 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1506491 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1506493 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1506495 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57015_bv%2Fj",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57015/bv/j",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 1506496 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57015_bv%2Fj",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57015/bv/j",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null}
   [junit4]   2> 1506497 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1506497 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1506498 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1506499 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1506499 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1506505 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 1506506 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1507494 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1507494 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1507495 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1507495 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1507496 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1507496 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/'
   [junit4]   2> 1507521 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1507537 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1507563 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 1507659 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1507660 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1507668 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1508107 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1508113 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1508116 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1508122 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1508160 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1508160 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1508160 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1508160 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1508161 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@40c31593
   [junit4]   2> 1508164 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/data
   [junit4]   2> 1508164 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1508164 WARN  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1508166 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/data/index
   [junit4]   2> 1508166 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=97.138671875, floorSegmentMB=0.48046875, forceMergeDeletesPctAllowed=12.020233300651107, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4037780296373946
   [junit4]   2> 1508170 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@57330c55),segFN=segments_1,generation=1}
   [junit4]   2> 1508170 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1508178 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1508178 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1508178 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1508178 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1508179 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1508179 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1508179 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1508179 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1508179 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1508181 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/data/
   [junit4]   2> 1508181 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1508185 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1508187 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1508188 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1508196 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1508212 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /config,/update/csv,/update/json/docs,/update,/admin/ping,/admin/logging,/admin/file,/admin/properties,/admin/luke,/admin/threads,/admin/mbeans,/get,/admin/segments,/admin/plugins,/schema,/replication,/admin/system,/update/json,standard
   [junit4]   2> 1508219 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1508225 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1508225 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1508227 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1508227 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1508227 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=40.935546875, floorSegmentMB=1.298828125, forceMergeDeletesPctAllowed=21.197036073493443, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1508230 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@57330c55),segFN=segments_1,generation=1}
   [junit4]   2> 1508230 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1508231 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@66c17951[collection1] main
   [junit4]   2> 1508231 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1508233 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1508233 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1508234 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1508234 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1508235 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1508236 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1508237 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1508237 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1508237 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1508239 INFO  (searcherExecutor-4327-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@66c17951[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1508248 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1508248 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1508248 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1508249 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1508577785382174720
   [junit4]   2> 1508251 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 3 ms to seed version buckets with highest version 1508577785382174720
   [junit4]   2> 1508251 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1508252 INFO  (coreLoadExecutor-4326-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1508253 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57015/bv/j collection:control_collection shard:shard1
   [junit4]   2> 1508255 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1508271 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94286111418023940-core_node1-n_0000000000
   [junit4]   2> 1508272 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1508277 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1508277 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1508277 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57015/bv/j/collection1/
   [junit4]   2> 1508278 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1508278 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57015/bv/j/collection1/ has no replicas
   [junit4]   2> 1508278 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1508279 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1508280 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1508282 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 1508278 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57015/bv/j/collection1/ shard1
   [junit4]   2> 1508285 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1508288 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1508289 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57015/bv/j",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1508393 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1508395 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1508442 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57015/bv/j/collection1/ and leader is http://127.0.0.1:57015/bv/j/collection1/
   [junit4]   2> 1508443 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57015/bv/j
   [junit4]   2> 1508443 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1508443 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1508443 INFO  (coreZkRegister-4320-thread-1-processing-s:shard1 n:127.0.0.1:57015_bv%2Fj r:core_node1 x:collection1 c:control_collection) [n:127.0.0.1:57015_bv%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1508445 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1508447 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57015_bv%2Fj",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57015/bv/j",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null} current state version: 3
   [junit4]   2> 1508449 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57015_bv%2Fj",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57015/bv/j",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null}
   [junit4]   2> 1508501 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1508508 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1508510 INFO  (zkCallback-1047-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74d9ec3f name:ZooKeeperConnection Watcher:127.0.0.1:55464/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1508510 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1508510 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1508510 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1508514 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1508552 INFO  (zkCallback-1047-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1508552 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1508554 INFO  (zkCallback-1047-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1508555 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1509067 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores/collection1
   [junit4]   2> 1509068 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001
   [junit4]   2> 1509076 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1509098 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@da65d44{/bv/j,null,AVAILABLE}
   [junit4]   2> 1509098 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4362d919{HTTP/1.1}{127.0.0.1:59219}
   [junit4]   2> 1509098 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.e.j.s.Server Started @1512469ms
   [junit4]   2> 1509099 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=59219, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores, hostContext=/bv/j}
   [junit4]   2> 1509099 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1509099 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/'
   [junit4]   2> 1509117 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1509119 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1509129 INFO  (zkCallback-1048-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75317318 name:ZooKeeperConnection Watcher:127.0.0.1:55464/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1509129 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1509130 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1509136 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1509136 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/solr.xml
   [junit4]   2> 1509156 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores
   [junit4]   2> 1509156 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoreContainer New CoreContainer 753145205
   [junit4]   2> 1509156 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/]
   [junit4]   2> 1509156 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/lib
   [junit4]   2> 1509157 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/lib).
   [junit4]   2> 1509169 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1509170 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1509171 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1509171 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1509171 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55464/solr
   [junit4]   2> 1509171 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1509171 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1509177 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1509177 INFO  (zkCallback-1050-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e958a2c name:ZooKeeperConnection Watcher:127.0.0.1:55464 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1509178 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1509179 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1509182 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1509182 INFO  (zkCallback-1051-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29f184d8 name:ZooKeeperConnection Watcher:127.0.0.1:55464/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1509182 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1509186 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1510190 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59219_bv%2Fj
   [junit4]   2> 1510191 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59219_bv%2Fj
   [junit4]   2> 1510193 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1510194 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94286111418023944-127.0.0.1:59219_bv%2Fj-n_0000000001
   [junit4]   2> 1510195 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000 to know if I could be the leader
   [junit4]   2> 1510195 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1510196 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1510197 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores
   [junit4]   2> 1510197 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores/collection1/, dataDir=data/, schema=schema.xml, transient=false, collection=collection1, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores/collection1, name=collection1, coreNodeName=, configSetProperties=configsetprops.json}
   [junit4]   2> 1510197 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores/collection1/
   [junit4]   2> 1510198 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1510204 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1510205 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1510206 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1510206 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1510207 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:59219_bv%2Fj",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:59219/bv/j",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null} current state version: 4
   [junit4]   2> 1510207 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:59219_bv%2Fj",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:59219/bv/j",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null}
   [junit4]   2> 1510208 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1510208 INFO  (OverseerStateUpdate-94286111418023940-127.0.0.1:57015_bv%2Fj-n_0000000000) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
   [junit4]   2> 1510210 INFO  (zkCallback-1047-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1510210 INFO  (zkCallback-1051-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1510210 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1510213 INFO  (zkCallback-1051-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 1510214 INFO  (zkCallback-1045-thread-1-processing-n:127.0.0.1:57015_bv%2Fj) [n:127.0.0.1:57015_bv%2Fj    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 1510215 INFO  (zkCallback-1047-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 1510222 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1510222 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[ECA2FE4A899C4B36]) [n:127.0.0.1:59219_bv%2Fj    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1511213 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1511213 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1511213 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1511214 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1511214 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1511215 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1511242 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1511251 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1511276 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 1511351 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1511353 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1511382 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1511709 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1511711 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1511712 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1511715 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1511769 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1511770 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1511770 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1511770 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1511770 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@40c31593
   [junit4]   2> 1511771 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj) [n:127.0.0.1:59219_bv%2Fj c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_ECA2FE4A899C4B36-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1511771 INFO  (coreLoadExecutor-4337-thread-1-processing-n:127.0.0.1:59219_bv%2Fj)

[...truncated too long message...]

trics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4179456 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ECA2FE4A899C4B36]-worker) [n:127.0.0.1:47646_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Aug 04, 2015 9:22:40 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {range_facet_i_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), multiDefault=PostingsFormat(name=Direct), _version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), text=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=FST50, a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l_dv=FST50, other_tl1=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_i1=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=es_US, timezone=Europe/Chisinau
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=124236672,total=523763712
   [junit4]   2> NOTE: All tests run in this JVM: [DistribCursorPagingTest, TestDefaultSimilarityFactory, TestIntervalFaceting, PathHierarchyTokenizerFactoryTest, ZkStateWriterTest, PrimUtilsTest, AutoCommitTest, ConcurrentDeleteAndCreateCollectionTest, TestLeaderElectionZkExpiry, DistributedDebugComponentTest, HdfsThreadLeakTest, TestSolrQueryParser, TestCustomSort, HighlighterMaxOffsetTest, TestReplicaProperties, DocValuesTest, TestUniqueKeyFieldResource, TestFieldSortValues, VMParamsZkACLAndCredentialsProvidersTest, SortByFunctionTest, SearchHandlerTest, TestPartialUpdateDeduplication, ConnectionManagerTest, TestPerFieldSimilarity, TestSolrXml, FileBasedSpellCheckerTest, TestTolerantSearch, TestFileDictionaryLookup, TestStressReorder, TestSchemaNameResource, SolrIndexSplitterTest, DistributedIntervalFacetingTest, SolrInfoMBeanTest, TestSchemaVersionResource, TestDefaultSearchFieldResource, DirectUpdateHandlerTest, TestLRUStatsCache, OverseerTest, TestExpandComponent, HardAutoCommitTest, TestCollationField, AddBlockUpdateTest, BlockDirectoryTest, TestPseudoReturnFields, PolyFieldTest, BasicDistributedZk2Test, TestRequestStatusCollectionAPI, HLLSerializationTest, SOLR749Test, TestSystemIdResolver, SimpleCollectionCreateDeleteTest, DirectSolrSpellCheckerTest, DOMUtilTest, TestAuthenticationFramework, LoggingHandlerTest, SolrCloudExampleTest, SuggestComponentTest, TestQueryUtils, ReplicationFactorTest, TestSolrQueryParserDefaultOperatorResource, DataDrivenBlockJoinTest, UnloadDistributedZkTest, TestSurroundQueryParser, TestFieldTypeResource, SoftAutoCommitTest, DistributedQueryElevationComponentTest, TestCloudInspectUtil, XmlUpdateRequestHandlerTest, OverseerCollectionProcessorTest, ExplicitHLLTest, RuleEngineTest, CachingDirectoryFactoryTest, TestConfigSets, LeaderElectionTest, InfoHandlerTest, StatsComponentTest, NotRequiredUniqueKeyTest, TestReload, SuggesterTest, TestQueryTypes, TestXIncludeConfig, AsyncMigrateRouteKeyTest, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestRandomFaceting, TestZkChroot, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, CacheHeaderTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestOmitPositions, MoreLikeThisHandlerTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, TestJmxIntegration, JSONWriterTest, QueryParsingTest, BinaryUpdateRequestHandlerTest, PingRequestHandlerTest, TestBinaryResponseWriter, UpdateParamsTest, TestQuerySenderNoQuery, ResponseLogComponentTest, BadComponentTest, TestStressRecovery, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, TestConfig, OutputWriterTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, DirectSolrConnectionTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, EchoParamsTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest, CustomCollectionTest, LeaderFailoverAfterPartitionTest, MultiThreadedOCPTest, SharedFSAutoReplicaFailoverTest, TestRebalanceLeaders, HdfsBasicDistributedZk2Test, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [522/522] on J2 in 1208.06s, 1 test, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: There were test failures: 522 suites, 2081 tests, 2 errors, 25 ignored (14 assumptions)

Total time: 194 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 1127844587 bytes
Compression is 0.0%
Took 3 min 22 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 917 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/917/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test

Error Message:
Server refused connection at: http://127.0.0.1:42831/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:42831/collection1
	at __randomizedtesting.SeedInfo.seed([A83DF285E2ABAFA7:2069CD5F4C57C25F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:567)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:365)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:98)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:966)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:941)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:54)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:45)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 48 more


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1438559313683,generation=2,filelist=[_83.cfe, _83.cfs, _83.si, _85.cfe, _85.cfs, _85.si, _86.fdt, _86.fdx, _86.fnm, _86.nvd, _86.nvm, _86.si, _86_BlockTreeOrds_0.doc, _86_BlockTreeOrds_0.tio, _86_BlockTreeOrds_0.tipo, _89.fdt, _89.fdx, _89.fnm, _89.nvd, _89.nvm, _89.si, _89_BlockTreeOrds_0.doc, _89_BlockTreeOrds_0.tio, _89_BlockTreeOrds_0.tipo, _8b.fdt, _8b.fdx, _8b.fnm, _8b.nvd, _8b.nvm, _8b.si, _8b_BlockTreeOrds_0.doc, _8b_BlockTreeOrds_0.tio, _8b_BlockTreeOrds_0.tipo, _8c.fdt, _8c.fdx, _8c.fnm, _8c.nvd, _8c.nvm, _8c.si, _8c_BlockTreeOrds_0.doc, _8c_BlockTreeOrds_0.tio, _8c_BlockTreeOrds_0.tipo, _8d.fdt, _8d.fdx, _8d.fnm, _8d.nvd, _8d.nvm, _8d.si, _8d_BlockTreeOrds_0.doc, _8d_BlockTreeOrds_0.tio, _8d_BlockTreeOrds_0.tipo, _8g.fdt, _8g.fdx, _8g.fnm, _8g.nvd, _8g.nvm, _8g.si, _8g_BlockTreeOrds_0.doc, _8g_BlockTreeOrds_0.tio, _8g_BlockTreeOrds_0.tipo, _8h.fdt, _8h.fdx, _8h.fnm, _8h.nvd, _8h.nvm, _8h.si, _8h_BlockTreeOrds_0.doc, _8h_BlockTreeOrds_0.tio, _8h_BlockTreeOrds_0.tipo, _8l.fdt, _8l.fdx, _8l.fnm, _8l.nvd, _8l.nvm, _8l.si, _8l_BlockTreeOrds_0.doc, _8l_BlockTreeOrds_0.tio, _8l_BlockTreeOrds_0.tipo, _8n.fdt, _8n.fdx, _8n.fnm, _8n.nvd, _8n.nvm, _8n.si, _8n_BlockTreeOrds_0.doc, _8n_BlockTreeOrds_0.tio, _8n_BlockTreeOrds_0.tipo, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.nvd, _8p.nvm, _8p.si, _8p_BlockTreeOrds_0.doc, _8p_BlockTreeOrds_0.tio, _8p_BlockTreeOrds_0.tipo, _8q.fdt, _8q.fdx, _8q.fnm, _8q.nvd, _8q.nvm, _8q.si, _8q_BlockTreeOrds_0.doc, _8q_BlockTreeOrds_0.tio, _8q_BlockTreeOrds_0.tipo, _8r.fdt, _8r.fdx, _8r.fnm, _8r.nvd, _8r.nvm, _8r.si, _8r_BlockTreeOrds_0.doc, _8r_BlockTreeOrds_0.tio, _8r_BlockTreeOrds_0.tipo, _8s.fdt, _8s.fdx, _8s.fnm, _8s.nvd, _8s.nvm, _8s.si, _8s_BlockTreeOrds_0.doc, _8s_BlockTreeOrds_0.tio, _8s_BlockTreeOrds_0.tipo, _8t.fdt, _8t.fdx, _8t.fnm, _8t.nvd, _8t.nvm, _8t.si, _8t_BlockTreeOrds_0.doc, _8t_BlockTreeOrds_0.tio, _8t_BlockTreeOrds_0.tipo, _8u.fdt, _8u.fdx, _8u.fnm, _8u.nvd, _8u.nvm, _8u.si, _8u_BlockTreeOrds_0.doc, _8u_BlockTreeOrds_0.tio, _8u_BlockTreeOrds_0.tipo, _8v.fdt, _8v.fdx, _8v.fnm, _8v.nvd, _8v.nvm, _8v.si, _8v_BlockTreeOrds_0.doc, _8v_BlockTreeOrds_0.tio, _8v_BlockTreeOrds_0.tipo, _8w.fdt, _8w.fdx, _8w.fnm, _8w.nvd, _8w.nvm, _8w.si, _8w_BlockTreeOrds_0.doc, _8w_BlockTreeOrds_0.tio, _8w_BlockTreeOrds_0.tipo, _8x.fdt, _8x.fdx, _8x.fnm, _8x.nvd, _8x.nvm, _8x.si, _8x_BlockTreeOrds_0.doc, _8x_BlockTreeOrds_0.tio, _8x_BlockTreeOrds_0.tipo, _8y.fdt, _8y.fdx, _8y.fnm, _8y.nvd, _8y.nvm, _8y.si, _8y_BlockTreeOrds_0.doc, _8y_BlockTreeOrds_0.tio, _8y_BlockTreeOrds_0.tipo, segments_2]}]> but was:<[{indexVersion=1438559313683,generation=2,filelist=[_83.cfe, _83.cfs, _83.si, _85.cfe, _85.cfs, _85.si, _86.fdt, _86.fdx, _86.fnm, _86.nvd, _86.nvm, _86.si, _86_BlockTreeOrds_0.doc, _86_BlockTreeOrds_0.tio, _86_BlockTreeOrds_0.tipo, _89.fdt, _89.fdx, _89.fnm, _89.nvd, _89.nvm, _89.si, _89_BlockTreeOrds_0.doc, _89_BlockTreeOrds_0.tio, _89_BlockTreeOrds_0.tipo, _8b.fdt, _8b.fdx, _8b.fnm, _8b.nvd, _8b.nvm, _8b.si, _8b_BlockTreeOrds_0.doc, _8b_BlockTreeOrds_0.tio, _8b_BlockTreeOrds_0.tipo, _8c.fdt, _8c.fdx, _8c.fnm, _8c.nvd, _8c.nvm, _8c.si, _8c_BlockTreeOrds_0.doc, _8c_BlockTreeOrds_0.tio, _8c_BlockTreeOrds_0.tipo, _8d.fdt, _8d.fdx, _8d.fnm, _8d.nvd, _8d.nvm, _8d.si, _8d_BlockTreeOrds_0.doc, _8d_BlockTreeOrds_0.tio, _8d_BlockTreeOrds_0.tipo, _8g.fdt, _8g.fdx, _8g.fnm, _8g.nvd, _8g.nvm, _8g.si, _8g_BlockTreeOrds_0.doc, _8g_BlockTreeOrds_0.tio, _8g_BlockTreeOrds_0.tipo, _8h.fdt, _8h.fdx, _8h.fnm, _8h.nvd, _8h.nvm, _8h.si, _8h_BlockTreeOrds_0.doc, _8h_BlockTreeOrds_0.tio, _8h_BlockTreeOrds_0.tipo, _8l.fdt, _8l.fdx, _8l.fnm, _8l.nvd, _8l.nvm, _8l.si, _8l_BlockTreeOrds_0.doc, _8l_BlockTreeOrds_0.tio, _8l_BlockTreeOrds_0.tipo, _8n.fdt, _8n.fdx, _8n.fnm, _8n.nvd, _8n.nvm, _8n.si, _8n_BlockTreeOrds_0.doc, _8n_BlockTreeOrds_0.tio, _8n_BlockTreeOrds_0.tipo, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.nvd, _8p.nvm, _8p.si, _8p_BlockTreeOrds_0.doc, _8p_BlockTreeOrds_0.tio, _8p_BlockTreeOrds_0.tipo, _8q.fdt, _8q.fdx, _8q.fnm, _8q.nvd, _8q.nvm, _8q.si, _8q_BlockTreeOrds_0.doc, _8q_BlockTreeOrds_0.tio, _8q_BlockTreeOrds_0.tipo, _8r.fdt, _8r.fdx, _8r.fnm, _8r.nvd, _8r.nvm, _8r.si, _8r_BlockTreeOrds_0.doc, _8r_BlockTreeOrds_0.tio, _8r_BlockTreeOrds_0.tipo, _8s.fdt, _8s.fdx, _8s.fnm, _8s.nvd, _8s.nvm, _8s.si, _8s_BlockTreeOrds_0.doc, _8s_BlockTreeOrds_0.tio, _8s_BlockTreeOrds_0.tipo, _8t.fdt, _8t.fdx, _8t.fnm, _8t.nvd, _8t.nvm, _8t.si, _8t_BlockTreeOrds_0.doc, _8t_BlockTreeOrds_0.tio, _8t_BlockTreeOrds_0.tipo, _8u.fdt, _8u.fdx, _8u.fnm, _8u.nvd, _8u.nvm, _8u.si, _8u_BlockTreeOrds_0.doc, _8u_BlockTreeOrds_0.tio, _8u_BlockTreeOrds_0.tipo, _8v.fdt, _8v.fdx, _8v.fnm, _8v.nvd, _8v.nvm, _8v.si, _8v_BlockTreeOrds_0.doc, _8v_BlockTreeOrds_0.tio, _8v_BlockTreeOrds_0.tipo, _8w.fdt, _8w.fdx, _8w.fnm, _8w.nvd, _8w.nvm, _8w.si, _8w_BlockTreeOrds_0.doc, _8w_BlockTreeOrds_0.tio, _8w_BlockTreeOrds_0.tipo, _8x.fdt, _8x.fdx, _8x.fnm, _8x.nvd, _8x.nvm, _8x.si, _8x_BlockTreeOrds_0.doc, _8x_BlockTreeOrds_0.tio, _8x_BlockTreeOrds_0.tipo, _8y.fdt, _8y.fdx, _8y.fnm, _8y.nvd, _8y.nvm, _8y.si, _8y_BlockTreeOrds_0.doc, _8y_BlockTreeOrds_0.tio, _8y_BlockTreeOrds_0.tipo, segments_2]}, {indexVersion=1438559313683,generation=3,filelist=[_83.cfe, _83.cfs, _83.si, _85.cfe, _85.cfs, _85.si, _8h.fdt, _8h.fdx, _8h.fnm, _8h.nvd, _8h.nvm, _8h.si, _8h_BlockTreeOrds_0.doc, _8h_BlockTreeOrds_0.tio, _8h_BlockTreeOrds_0.tipo, _8l.fdt, _8l.fdx, _8l.fnm, _8l.nvd, _8l.nvm, _8l.si, _8l_BlockTreeOrds_0.doc, _8l_BlockTreeOrds_0.tio, _8l_BlockTreeOrds_0.tipo, _8n.fdt, _8n.fdx, _8n.fnm, _8n.nvd, _8n.nvm, _8n.si, _8n_BlockTreeOrds_0.doc, _8n_BlockTreeOrds_0.tio, _8n_BlockTreeOrds_0.tipo, _8y.fdt, _8y.fdx, _8y.fnm, _8y.nvd, _8y.nvm, _8y.si, _8y_BlockTreeOrds_0.doc, _8y_BlockTreeOrds_0.tio, _8y_BlockTreeOrds_0.tipo, _8z.fdt, _8z.fdx, _8z.fnm, _8z.nvd, _8z.nvm, _8z.si, _8z_BlockTreeOrds_0.doc, _8z_BlockTreeOrds_0.tio, _8z_BlockTreeOrds_0.tipo, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1438559313683,generation=2,filelist=[_83.cfe, _83.cfs, _83.si, _85.cfe, _85.cfs, _85.si, _86.fdt, _86.fdx, _86.fnm, _86.nvd, _86.nvm, _86.si, _86_BlockTreeOrds_0.doc, _86_BlockTreeOrds_0.tio, _86_BlockTreeOrds_0.tipo, _89.fdt, _89.fdx, _89.fnm, _89.nvd, _89.nvm, _89.si, _89_BlockTreeOrds_0.doc, _89_BlockTreeOrds_0.tio, _89_BlockTreeOrds_0.tipo, _8b.fdt, _8b.fdx, _8b.fnm, _8b.nvd, _8b.nvm, _8b.si, _8b_BlockTreeOrds_0.doc, _8b_BlockTreeOrds_0.tio, _8b_BlockTreeOrds_0.tipo, _8c.fdt, _8c.fdx, _8c.fnm, _8c.nvd, _8c.nvm, _8c.si, _8c_BlockTreeOrds_0.doc, _8c_BlockTreeOrds_0.tio, _8c_BlockTreeOrds_0.tipo, _8d.fdt, _8d.fdx, _8d.fnm, _8d.nvd, _8d.nvm, _8d.si, _8d_BlockTreeOrds_0.doc, _8d_BlockTreeOrds_0.tio, _8d_BlockTreeOrds_0.tipo, _8g.fdt, _8g.fdx, _8g.fnm, _8g.nvd, _8g.nvm, _8g.si, _8g_BlockTreeOrds_0.doc, _8g_BlockTreeOrds_0.tio, _8g_BlockTreeOrds_0.tipo, _8h.fdt, _8h.fdx, _8h.fnm, _8h.nvd, _8h.nvm, _8h.si, _8h_BlockTreeOrds_0.doc, _8h_BlockTreeOrds_0.tio, _8h_BlockTreeOrds_0.tipo, _8l.fdt, _8l.fdx, _8l.fnm, _8l.nvd, _8l.nvm, _8l.si, _8l_BlockTreeOrds_0.doc, _8l_BlockTreeOrds_0.tio, _8l_BlockTreeOrds_0.tipo, _8n.fdt, _8n.fdx, _8n.fnm, _8n.nvd, _8n.nvm, _8n.si, _8n_BlockTreeOrds_0.doc, _8n_BlockTreeOrds_0.tio, _8n_BlockTreeOrds_0.tipo, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.nvd, _8p.nvm, _8p.si, _8p_BlockTreeOrds_0.doc, _8p_BlockTreeOrds_0.tio, _8p_BlockTreeOrds_0.tipo, _8q.fdt, _8q.fdx, _8q.fnm, _8q.nvd, _8q.nvm, _8q.si, _8q_BlockTreeOrds_0.doc, _8q_BlockTreeOrds_0.tio, _8q_BlockTreeOrds_0.tipo, _8r.fdt, _8r.fdx, _8r.fnm, _8r.nvd, _8r.nvm, _8r.si, _8r_BlockTreeOrds_0.doc, _8r_BlockTreeOrds_0.tio, _8r_BlockTreeOrds_0.tipo, _8s.fdt, _8s.fdx, _8s.fnm, _8s.nvd, _8s.nvm, _8s.si, _8s_BlockTreeOrds_0.doc, _8s_BlockTreeOrds_0.tio, _8s_BlockTreeOrds_0.tipo, _8t.fdt, _8t.fdx, _8t.fnm, _8t.nvd, _8t.nvm, _8t.si, _8t_BlockTreeOrds_0.doc, _8t_BlockTreeOrds_0.tio, _8t_BlockTreeOrds_0.tipo, _8u.fdt, _8u.fdx, _8u.fnm, _8u.nvd, _8u.nvm, _8u.si, _8u_BlockTreeOrds_0.doc, _8u_BlockTreeOrds_0.tio, _8u_BlockTreeOrds_0.tipo, _8v.fdt, _8v.fdx, _8v.fnm, _8v.nvd, _8v.nvm, _8v.si, _8v_BlockTreeOrds_0.doc, _8v_BlockTreeOrds_0.tio, _8v_BlockTreeOrds_0.tipo, _8w.fdt, _8w.fdx, _8w.fnm, _8w.nvd, _8w.nvm, _8w.si, _8w_BlockTreeOrds_0.doc, _8w_BlockTreeOrds_0.tio, _8w_BlockTreeOrds_0.tipo, _8x.fdt, _8x.fdx, _8x.fnm, _8x.nvd, _8x.nvm, _8x.si, _8x_BlockTreeOrds_0.doc, _8x_BlockTreeOrds_0.tio, _8x_BlockTreeOrds_0.tipo, _8y.fdt, _8y.fdx, _8y.fnm, _8y.nvd, _8y.nvm, _8y.si, _8y_BlockTreeOrds_0.doc, _8y_BlockTreeOrds_0.tio, _8y_BlockTreeOrds_0.tipo, segments_2]}]> but was:<[{indexVersion=1438559313683,generation=2,filelist=[_83.cfe, _83.cfs, _83.si, _85.cfe, _85.cfs, _85.si, _86.fdt, _86.fdx, _86.fnm, _86.nvd, _86.nvm, _86.si, _86_BlockTreeOrds_0.doc, _86_BlockTreeOrds_0.tio, _86_BlockTreeOrds_0.tipo, _89.fdt, _89.fdx, _89.fnm, _89.nvd, _89.nvm, _89.si, _89_BlockTreeOrds_0.doc, _89_BlockTreeOrds_0.tio, _89_BlockTreeOrds_0.tipo, _8b.fdt, _8b.fdx, _8b.fnm, _8b.nvd, _8b.nvm, _8b.si, _8b_BlockTreeOrds_0.doc, _8b_BlockTreeOrds_0.tio, _8b_BlockTreeOrds_0.tipo, _8c.fdt, _8c.fdx, _8c.fnm, _8c.nvd, _8c.nvm, _8c.si, _8c_BlockTreeOrds_0.doc, _8c_BlockTreeOrds_0.tio, _8c_BlockTreeOrds_0.tipo, _8d.fdt, _8d.fdx, _8d.fnm, _8d.nvd, _8d.nvm, _8d.si, _8d_BlockTreeOrds_0.doc, _8d_BlockTreeOrds_0.tio, _8d_BlockTreeOrds_0.tipo, _8g.fdt, _8g.fdx, _8g.fnm, _8g.nvd, _8g.nvm, _8g.si, _8g_BlockTreeOrds_0.doc, _8g_BlockTreeOrds_0.tio, _8g_BlockTreeOrds_0.tipo, _8h.fdt, _8h.fdx, _8h.fnm, _8h.nvd, _8h.nvm, _8h.si, _8h_BlockTreeOrds_0.doc, _8h_BlockTreeOrds_0.tio, _8h_BlockTreeOrds_0.tipo, _8l.fdt, _8l.fdx, _8l.fnm, _8l.nvd, _8l.nvm, _8l.si, _8l_BlockTreeOrds_0.doc, _8l_BlockTreeOrds_0.tio, _8l_BlockTreeOrds_0.tipo, _8n.fdt, _8n.fdx, _8n.fnm, _8n.nvd, _8n.nvm, _8n.si, _8n_BlockTreeOrds_0.doc, _8n_BlockTreeOrds_0.tio, _8n_BlockTreeOrds_0.tipo, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.nvd, _8p.nvm, _8p.si, _8p_BlockTreeOrds_0.doc, _8p_BlockTreeOrds_0.tio, _8p_BlockTreeOrds_0.tipo, _8q.fdt, _8q.fdx, _8q.fnm, _8q.nvd, _8q.nvm, _8q.si, _8q_BlockTreeOrds_0.doc, _8q_BlockTreeOrds_0.tio, _8q_BlockTreeOrds_0.tipo, _8r.fdt, _8r.fdx, _8r.fnm, _8r.nvd, _8r.nvm, _8r.si, _8r_BlockTreeOrds_0.doc, _8r_BlockTreeOrds_0.tio, _8r_BlockTreeOrds_0.tipo, _8s.fdt, _8s.fdx, _8s.fnm, _8s.nvd, _8s.nvm, _8s.si, _8s_BlockTreeOrds_0.doc, _8s_BlockTreeOrds_0.tio, _8s_BlockTreeOrds_0.tipo, _8t.fdt, _8t.fdx, _8t.fnm, _8t.nvd, _8t.nvm, _8t.si, _8t_BlockTreeOrds_0.doc, _8t_BlockTreeOrds_0.tio, _8t_BlockTreeOrds_0.tipo, _8u.fdt, _8u.fdx, _8u.fnm, _8u.nvd, _8u.nvm, _8u.si, _8u_BlockTreeOrds_0.doc, _8u_BlockTreeOrds_0.tio, _8u_BlockTreeOrds_0.tipo, _8v.fdt, _8v.fdx, _8v.fnm, _8v.nvd, _8v.nvm, _8v.si, _8v_BlockTreeOrds_0.doc, _8v_BlockTreeOrds_0.tio, _8v_BlockTreeOrds_0.tipo, _8w.fdt, _8w.fdx, _8w.fnm, _8w.nvd, _8w.nvm, _8w.si, _8w_BlockTreeOrds_0.doc, _8w_BlockTreeOrds_0.tio, _8w_BlockTreeOrds_0.tipo, _8x.fdt, _8x.fdx, _8x.fnm, _8x.nvd, _8x.nvm, _8x.si, _8x_BlockTreeOrds_0.doc, _8x_BlockTreeOrds_0.tio, _8x_BlockTreeOrds_0.tipo, _8y.fdt, _8y.fdx, _8y.fnm, _8y.nvd, _8y.nvm, _8y.si, _8y_BlockTreeOrds_0.doc, _8y_BlockTreeOrds_0.tio, _8y_BlockTreeOrds_0.tipo, segments_2]}, {indexVersion=1438559313683,generation=3,filelist=[_83.cfe, _83.cfs, _83.si, _85.cfe, _85.cfs, _85.si, _8h.fdt, _8h.fdx, _8h.fnm, _8h.nvd, _8h.nvm, _8h.si, _8h_BlockTreeOrds_0.doc, _8h_BlockTreeOrds_0.tio, _8h_BlockTreeOrds_0.tipo, _8l.fdt, _8l.fdx, _8l.fnm, _8l.nvd, _8l.nvm, _8l.si, _8l_BlockTreeOrds_0.doc, _8l_BlockTreeOrds_0.tio, _8l_BlockTreeOrds_0.tipo, _8n.fdt, _8n.fdx, _8n.fnm, _8n.nvd, _8n.nvm, _8n.si, _8n_BlockTreeOrds_0.doc, _8n_BlockTreeOrds_0.tio, _8n_BlockTreeOrds_0.tipo, _8y.fdt, _8y.fdx, _8y.fnm, _8y.nvd, _8y.nvm, _8y.si, _8y_BlockTreeOrds_0.doc, _8y_BlockTreeOrds_0.tio, _8y_BlockTreeOrds_0.tipo, _8z.fdt, _8z.fdx, _8z.fnm, _8z.nvd, _8z.nvm, _8z.si, _8z_BlockTreeOrds_0.doc, _8z_BlockTreeOrds_0.tio, _8z_BlockTreeOrds_0.tipo, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([A83DF285E2ABAFA7:8DEAE9B592E3A1A4]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2288, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2288, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59057: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([A83DF285E2ABAFA7]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)




Build Log:
[...truncated 10429 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/init-core-data-001
   [junit4]   2> 89758 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 89905 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 89924 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 89928 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 89945 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_34246_hdfs____j907h9/webapp
   [junit4]   2> 90136 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 90726 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34246
   [junit4]   2> 91100 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 91102 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 91129 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53342_datanode____.t6x4hy/webapp
   [junit4]   2> 91299 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 91893 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53342
   [junit4]   2> 92104 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 92106 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 92125 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_52923_datanode____.1cj7do/webapp
   [junit4]   2> 92225 INFO  (IPC Server handler 2 on 44409) [    ] BlockStateChange BLOCK* processReport: from storage DS-35d05124-4777-4665-8172-2d0e34e53131 node DatanodeRegistration(127.0.0.1, datanodeUuid=b6b82feb-1657-4492-99ff-873a3dc06567, infoPort=53342, ipcPort=41792, storageInfo=lv=-56;cid=testClusterID;nsid=1035212670;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 92226 INFO  (IPC Server handler 2 on 44409) [    ] BlockStateChange BLOCK* processReport: from storage DS-a9d935ef-91a1-4a23-91a6-9344899ac28a node DatanodeRegistration(127.0.0.1, datanodeUuid=b6b82feb-1657-4492-99ff-873a3dc06567, infoPort=53342, ipcPort=41792, storageInfo=lv=-56;cid=testClusterID;nsid=1035212670;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 92366 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 92810 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[A83DF285E2ABAFA7]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52923
   [junit4]   2> 93239 INFO  (IPC Server handler 7 on 44409) [    ] BlockStateChange BLOCK* processReport: from storage DS-7d5bd885-927f-4fc8-9d7c-9170038bf9c5 node DatanodeRegistration(127.0.0.1, datanodeUuid=7b582bca-2606-4e27-8b48-ab2dbab5d0d4, infoPort=52923, ipcPort=58121, storageInfo=lv=-56;cid=testClusterID;nsid=1035212670;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 93240 INFO  (IPC Server handler 7 on 44409) [    ] BlockStateChange BLOCK* processReport: from storage DS-208001a0-7230-4b95-bc92-d13706682224 node DatanodeRegistration(127.0.0.1, datanodeUuid=7b582bca-2606-4e27-8b48-ab2dbab5d0d4, infoPort=52923, ipcPort=58121, storageInfo=lv=-56;cid=testClusterID;nsid=1035212670;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 93417 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 93417 INFO  (Thread-449) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 93417 INFO  (Thread-449) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 93521 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.ZkTestServer start zk server on port:41971
   [junit4]   2> 93521 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 93536 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 93537 INFO  (zkCallback-59-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8d4d1c name:ZooKeeperConnection Watcher:127.0.0.1:41971 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 93538 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 93538 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 93538 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 93541 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 93569 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 93571 INFO  (zkCallback-60-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1eebbb14 name:ZooKeeperConnection Watcher:127.0.0.1:41971/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 93571 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 93572 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 93572 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 93574 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 93576 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 93578 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 93580 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 93581 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 93584 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 93584 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 93587 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 93587 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 93589 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 93589 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 93591 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 93592 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 93594 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 93594 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 93596 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 93596 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 93598 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 93598 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 93600 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 93600 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 93602 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 93602 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 93604 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 93604 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 94062 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores/collection1
   [junit4]   2> 94064 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 94067 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@db81c27{/,null,AVAILABLE}
   [junit4]   2> 94067 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.e.j.s.ServerConnector Started ServerConnector@32404b7a{HTTP/1.1}{127.0.0.1:44349}
   [junit4]   2> 94068 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.e.j.s.Server Started @97509ms
   [junit4]   2> 94068 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores, solr.data.dir=hdfs://localhost:44409/hdfs__localhost_44409__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001_tempDir-002_control_data, hostContext=/, hostPort=44349}
   [junit4]   2> 94069 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 94069 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/'
   [junit4]   2> 94092 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 94154 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 94156 INFO  (zkCallback-61-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3523a09e name:ZooKeeperConnection Watcher:127.0.0.1:41971/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 94163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 94164 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 94169 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 94170 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/solr.xml
   [junit4]   2> 94208 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores
   [junit4]   2> 94209 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.CoreContainer New CoreContainer 542720985
   [junit4]   2> 94209 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/]
   [junit4]   2> 94209 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/lib
   [junit4]   2> 94209 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/lib).
   [junit4]   2> 94219 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 94220 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 94221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 94221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 94221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41971/solr
   [junit4]   2> 94222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 94222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 94223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 94224 INFO  (zkCallback-63-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38f91517 name:ZooKeeperConnection Watcher:127.0.0.1:41971 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 94224 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 94225 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 94230 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 94249 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@573582f1 name:ZooKeeperConnection Watcher:127.0.0.1:41971/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 94249 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 94257 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 94262 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 94265 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 94267 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 94270 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 94272 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 94274 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 94276 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 94277 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44349_
   [junit4]   2> 94278 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44349_
   [junit4]   2> 94279 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 94281 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 94282 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 94283 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94277292842418180-127.0.0.1:44349_-n_0000000000
   [junit4]   2> 94284 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44349_
   [junit4]   2> 94284 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 94286 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.Overseer Overseer (id=94277292842418180-127.0.0.1:44349_-n_0000000000) starting
   [junit4]   2> 94294 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 94321 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 94322 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 94333 INFO  (OverseerCollectionProcessor-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 94347 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 94351 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 94351 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 94352 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores
   [junit4]   2> 94353 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores/collection1/, loadOnStartup=true, coreNodeName=, name=collection1, collection=control_collection, configSetProperties=configsetprops.json, dataDir=data/, transient=false, schema=schema.xml, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores/collection1, config=solrconfig.xml}
   [junit4]   2> 94354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores/collection1/
   [junit4]   2> 94354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 94366 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 94366 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 94367 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 94369 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 94371 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 94371 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [n:127.0.0.1:44349_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 94372 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44349_",
   [junit4]   2>   "base_url":"http://127.0.0.1:44349",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 94373 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44349_",
   [junit4]   2>   "base_url":"http://127.0.0.1:44349",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 94373 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 94373 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 94375 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 94377 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 95368 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 95368 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 95369 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 95369 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 95370 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 95370 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores/collection1/'
   [junit4]   2> 95393 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 95504 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 95526 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 95560 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 95562 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 95595 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 96147 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 96159 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 96161 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 96236 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 96241 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 96245 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 96246 WARN  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 96246 WARN  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 96246 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 96247 WARN  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 96247 WARN  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 96248 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 96249 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 96249 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 96249 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44409/solr_hdfs_home
   [junit4]   2> 96249 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 96250 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_A83DF285E2ABAFA7-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 96250 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34593f0a
   [junit4]   2> 96250 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 96286 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 96286 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 96288 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 96300 WARN  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:44409/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 96301 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 96311 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 96311 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 96318 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 96318 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 96319 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 96319 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=82.763671875, floorSegmentMB=1.251953125, forceMergeDeletesPctAllowed=10.716497520217752, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 96508 INFO  (IPC Server handler 8 on 44409) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60595 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7d5bd885-927f-4fc8-9d7c-9170038bf9c5:NORMAL:127.0.0.1:40698|RBW], ReplicaUnderConstruction[[DISK]DS-35d05124-4777-4665-8172-2d0e34e53131:NORMAL:127.0.0.1:60595|RBW]]} size 0
   [junit4]   2> 96512 INFO  (IPC Server handler 0 on 44409) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40698 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7d5bd885-927f-4fc8-9d7c-9170038bf9c5:NORMAL:127.0.0.1:40698|RBW], ReplicaUnderConstruction[[DISK]DS-35d05124-4777-4665-8172-2d0e34e53131:NORMAL:127.0.0.1:60595|RBW]]} size 0
   [junit4]   2> 96520 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b75202ab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@122bc363); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 96520 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 96523 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:44409/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 96532 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 96536 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 96537 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 96537 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 96537 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 96537 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 96538 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 96538 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 96538 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 96544 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 96545 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 96547 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 96548 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 96555 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /get,/update,standard,/admin/segments,/admin/luke,/update/json,/admin/file,/admin/ping,/admin/plugins,/update/json/docs,/replication,/admin/properties,/admin/threads,/schema,/admin/logging,/admin/system,/admin/mbeans,/update/csv,/config
   [junit4]   2> 96557 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 96558 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 96566 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 96585 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 96585 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 96597 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1b49af42
   [junit4]   2> 96626 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b75202ab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@122bc363); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 96626 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 96628 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3f9643fc[collection1] main
   [junit4]   2> 96628 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 96629 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 96629 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 96630 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 96630 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 96630 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 96631 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 96631 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 96631 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 96631 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 96634 INFO  (searcherExecutor-156-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3f9643fc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 96639 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 96639 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 96639 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 96639 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1508436688855105536
   [junit4]   2> 96640 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1 ms to seed version buckets with highest version 1508436688855105536
   [junit4]   2> 96640 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 96641 INFO  (coreLoadExecutor-155-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 96650 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44349 collection:control_collection shard:shard1
   [junit4]   2> 96663 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 96668 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94277292842418180-core_node1-n_0000000000
   [junit4]   2> 96669 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 96671 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 96672 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 96674 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 96677 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 96679 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 96679 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 96679 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44349/collection1/
   [junit4]   2> 96679 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 96679 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44349/collection1/ has no replicas
   [junit4]   2> 96680 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44349/collection1/ shard1
   [junit4]   2> 96680 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 96684 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 96685 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44349",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 96789 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 96792 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 96838 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44349/collection1/ and leader is http://127.0.0.1:44349/collection1/
   [junit4]   2> 96838 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44349
   [junit4]   2> 96839 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 96839 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 96839 INFO  (coreZkRegister-149-thread-1-processing-c:control_collection x:collection1 n:127.0.0.1:44349_ r:core_node1 s:shard1) [n:127.0.0.1:44349_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 96850 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 96852 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44349_",
   [junit4]   2>   "base_url":"http://127.0.0.1:44349",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 96853 INFO  (OverseerStateUpdate-94277292842418180-127.0.0.1:44349_-n_0000000000) [n:127.0.0.1:44349_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44349_",
   [junit4]   2>   "base_url":"http://127.0.0.1:44349",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 96874 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 96882 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 96885 INFO  (zkCallback-66-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36170b49 name:ZooKeeperConnection Watcher:127.0.0.1:41971/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 96885 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 96887 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 96887 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 96891 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[A83DF285E2ABAFA7]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 96958 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:44349_) [n:127.0.0.1:44349_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:

[...truncated too long message...]

ce/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150802_191347_892.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20150802_191347_892.suites
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 842 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 23109110163 bytes
Compression is 0.0%
Took 1 hr 0 min
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 916 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/916/

4 tests failed.
FAILED:  org.apache.solr.DistributedIntervalFacetingTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=893, name=Thread-280, state=RUNNABLE, group=TGRP-DistributedIntervalFacetingTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=893, name=Thread-280, state=RUNNABLE, group=TGRP-DistributedIntervalFacetingTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:48307/b_s/collection1
	at __randomizedtesting.SeedInfo.seed([D2CDF61640F93AEA]:0)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:632)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:48307/b_s/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:627)
Caused by: java.io.InterruptedIOException: Connection has been shut down
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:579)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:54)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:45)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 6 more
Caused by: org.apache.http.impl.conn.ConnectionShutdownException
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.ensurePoolEntry(ManagedClientConnectionImpl.java:123)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.getRoute(ManagedClientConnectionImpl.java:282)
	at org.apache.http.client.protocol.RequestProxyAuthentication.process(RequestProxyAuthentication.java:74)
	at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
	at org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
	... 11 more


FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Captured an uncaught exception in thread: Thread[id=45119, name=Thread-44333, state=RUNNABLE, group=TGRP-TestDistributedSearch]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=45119, name=Thread-44333, state=RUNNABLE, group=TGRP-TestDistributedSearch]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:55624//collection1
	at __randomizedtesting.SeedInfo.seed([D2CDF61640F93AEA]:0)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:632)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:55624//collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:627)
Caused by: java.io.InterruptedIOException: Connection has been shut down
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:579)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:54)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:45)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 6 more
Caused by: org.apache.http.impl.conn.ConnectionShutdownException
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.ensurePoolEntry(ManagedClientConnectionImpl.java:123)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.getRoute(ManagedClientConnectionImpl.java:282)
	at org.apache.http.client.protocol.RequestProxyAuthentication.process(RequestProxyAuthentication.java:74)
	at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
	at org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
	... 11 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=21285, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21285, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:49585: collection already exists: awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([D2CDF61640F93AEA]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)


FAILED:  org.apache.solr.handler.component.TermVectorComponentDistributedTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=23532, name=Thread-17722, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=23532, name=Thread-17722, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
	at __randomizedtesting.SeedInfo.seed([D2CDF61640F93AEA:5A99C9CCEE055712]:0)
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:60201/p/fi/collection1
	at __randomizedtesting.SeedInfo.seed([D2CDF61640F93AEA]:0)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:632)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:60201/p/fi/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:627)
Caused by: java.io.InterruptedIOException: Connection has been shut down
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:579)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:54)
	at org.apache.solr.util.SolrHttpClient$SolrSystemDefaultHttpClient.execute(SolrHttpClient.java:45)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:465)
	... 6 more
Caused by: org.apache.http.impl.conn.ConnectionShutdownException
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.ensureConnection(ManagedClientConnectionImpl.java:115)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.getSSLSession(ManagedClientConnectionImpl.java:247)
	at org.apache.http.impl.client.DefaultUserTokenHandler.getUserToken(DefaultUserTokenHandler.java:81)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:550)
	... 11 more




Build Log:
[...truncated 10326 lines...]
   [junit4] Suite: org.apache.solr.DistributedIntervalFacetingTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/init-core-data-001
   [junit4]   2> 223796 INFO  (SUITE-DistributedIntervalFacetingTest-seed#[D2CDF61640F93AEA]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 223796 INFO  (SUITE-DistributedIntervalFacetingTest-seed#[D2CDF61640F93AEA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /b_s/
   [junit4]   2> 224186 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1
   [junit4]   2> 224188 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 224190 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7bc17541{/b_s,null,AVAILABLE}
   [junit4]   2> 224197 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@59f93de0{HTTP/1.1}{127.0.0.1:47709}
   [junit4]   2> 224197 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.Server Started @227255ms
   [junit4]   2> 224197 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {schema=schema-distrib-interval-faceting.xml, solrconfig=solrconfig-basic.xml, hostContext=/b_s, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores, hostPort=47709}
   [junit4]   2> 224198 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 224198 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/'
   [junit4]   2> 224213 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/solr.xml
   [junit4]   2> 224228 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores
   [junit4]   2> 224228 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer New CoreContainer 1232744362
   [junit4]   2> 224228 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/]
   [junit4]   2> 224228 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/lib
   [junit4]   2> 224229 WARN  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/lib).
   [junit4]   2> 224238 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 224239 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 224240 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 224240 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 224240 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 224242 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores
   [junit4]   2> 224243 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/, config=solrconfig-basic.xml, configSetProperties=configsetprops.json, name=collection1, configSet=collection1, dataDir=data/, schema=schema-distrib-interval-faceting.xml, loadOnStartup=true, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1}
   [junit4]   2> 224243 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/
   [junit4]   2> 224243 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 224244 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 224251 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 224252 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 224253 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 224253 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 224253 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 224276 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 224287 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 224316 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-basic.xml
   [junit4]   2> 224316 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema-distrib-interval-faceting.xml
   [junit4]   2> 224321 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 224337 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is id
   [junit4]   2> 224339 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 224340 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 224340 INFO  (coreLoadExecutor-203-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 224341 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 224341 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 224341 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 224342 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/data
   [junit4]   2> 224342 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/data/index/
   [junit4]   2> 224342 WARN  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 224343 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/data/index
   [junit4]   2> 224343 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=6, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.798828125, noCFSRatio=1.0]
   [junit4]   2> 224344 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@448f3ab0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17561bfa; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_1,generation=1}
   [junit4]   2> 224344 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 224345 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/data/
   [junit4]   2> 224345 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/control/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 224347 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 224350 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 224351 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 224352 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 224353 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 224355 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/luke,/admin/plugins,/replication,/get,/admin/segments,/admin/system,/update/json,/admin/ping,/update/json/docs,/schema,/update/csv,/admin/mbeans,standard,/admin/file,/admin/logging,/update,/admin/threads,/admin/properties,/config
   [junit4]   2> 224356 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 224357 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 224357 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 224357 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=46.2958984375, floorSegmentMB=0.626953125, forceMergeDeletesPctAllowed=9.03776042347282, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 224358 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@448f3ab0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17561bfa; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_1,generation=1}
   [junit4]   2> 224358 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 224358 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6401b48a[collection1] main
   [junit4]   2> 224359 WARN  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 224359 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 224359 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 224359 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 224359 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 224359 INFO  (coreLoadExecutor-203-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 224360 INFO  (searcherExecutor-204-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6401b48a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 225073 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1
   [junit4]   2> 225074 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 225080 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1bb70a88{/b_s,null,AVAILABLE}
   [junit4]   2> 225081 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@e1474ef{HTTP/1.1}{127.0.0.1:48307}
   [junit4]   2> 225081 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.e.j.s.Server Started @228139ms
   [junit4]   2> 225082 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig-basic.xml, hostContext=/b_s, schema=schema-distrib-interval-faceting.xml, hostPort=48307, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores}
   [junit4]   2> 225082 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 225082 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/'
   [junit4]   2> 225102 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/solr.xml
   [junit4]   2> 225126 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores
   [junit4]   2> 225127 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer New CoreContainer 1326342700
   [junit4]   2> 225127 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/]
   [junit4]   2> 225128 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/lib
   [junit4]   2> 225129 WARN  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/lib).
   [junit4]   2> 225145 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 225149 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 225150 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 225150 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 225150 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 225152 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores
   [junit4]   2> 225153 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSet=collection1, dataDir=data/, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/, config=solrconfig-basic.xml, name=collection1, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1, schema=schema-distrib-interval-faceting.xml, configSetProperties=configsetprops.json}
   [junit4]   2> 225153 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/
   [junit4]   2> 225154 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 225175 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 225175 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 225186 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 225186 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 225187 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 225187 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 225221 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 225253 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 225315 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig-basic.xml
   [junit4]   2> 225316 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema-distrib-interval-faceting.xml
   [junit4]   2> 225325 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 225351 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is id
   [junit4]   2> 225352 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 225354 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf'
   [junit4]   2> 225354 INFO  (coreLoadExecutor-213-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 225354 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 225354 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 225354 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 225355 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/data
   [junit4]   2> 225355 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/data/index/
   [junit4]   2> 225355 WARN  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 225356 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 225356 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=6, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.798828125, noCFSRatio=1.0]
   [junit4]   2> 225357 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@22e82fb1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39931f5e; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_1,generation=1}
   [junit4]   2> 225357 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 225358 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/data/
   [junit4]   2> 225359 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.DistributedIntervalFacetingTest_D2CDF61640F93AEA-001/tempDir-001/shard0/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 225361 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 225364 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 225366 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 225367 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 225368 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 225374 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/luke,/admin/plugins,/replication,/get,/admin/segments,/admin/system,/update/json,/admin/ping,/update/json/docs,/schema,/update/csv,/admin/mbeans,standard,/admin/file,/admin/logging,/update,/admin/threads,/admin/properties,/config
   [junit4]   2> 225375 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 225375 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 225376 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 225376 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=46.2958984375, floorSegmentMB=0.626953125, forceMergeDeletesPctAllowed=9.03776042347282, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 225377 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@22e82fb1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39931f5e; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_1,generation=1}
   [junit4]   2> 225377 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 225377 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2080973e[collection1] main
   [junit4]   2> 225377 WARN  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 225378 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 225378 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 225378 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 225378 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 225378 INFO  (coreLoadExecutor-213-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 225379 INFO  (searcherExecutor-214-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2080973e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 225687 INFO  (TEST-DistributedIntervalFacetingTest.test-seed#[D2CDF61640F93AEA]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 225692 INFO  (qtp483398844-797) [    x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 225692 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
   [junit4]   2> 225695 INFO  (qtp83200478-815) [    x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 225696 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
   [junit4]   2> 225699 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 225701 INFO  (qtp483398844-798) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@448f3ab0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17561bfa; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@448f3ab0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17561bfa; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_2,generation=2}
   [junit4]   2> 225701 INFO  (qtp483398844-798) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 225702 INFO  (qtp483398844-798) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@147b5907[collection1] main
   [junit4]   2> 225702 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 225702 INFO  (searcherExecutor-204-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@147b5907[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 225702 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={softCommit=false&waitSearcher=true&commit=true&wt=javabin&version=2} {commit=} 0 3
   [junit4]   2> 225706 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 225707 INFO  (qtp83200478-816) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@22e82fb1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39931f5e; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@22e82fb1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39931f5e; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_2,generation=2}
   [junit4]   2> 225707 INFO  (qtp83200478-816) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 225707 INFO  (qtp83200478-816) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6b8dd9da[collection1] main
   [junit4]   2> 225708 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 225710 INFO  (searcherExecutor-214-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6b8dd9da[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 225710 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={softCommit=false&waitSearcher=true&commit=true&wt=javabin&version=2} {commit=} 0 4
   [junit4]   2> 225723 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[0]} 0 8
   [junit4]   2> 225730 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[0]} 0 3
   [junit4]   2> 225737 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[1]} 0 3
   [junit4]   2> 225742 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[1]} 0 1
   [junit4]   2> 225748 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[2]} 0 1
   [junit4]   2> 225755 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[2]} 0 3
   [junit4]   2> 225762 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[3]} 0 1
   [junit4]   2> 225768 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[3]} 0 1
   [junit4]   2> 225773 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[4]} 0 1
   [junit4]   2> 225781 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[4]} 0 2
   [junit4]   2> 225786 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[5]} 0 1
   [junit4]   2> 225795 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[5]} 0 1
   [junit4]   2> 225799 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[6]} 0 1
   [junit4]   2> 225803 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[6]} 0 1
   [junit4]   2> 225808 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[7]} 0 1
   [junit4]   2> 225812 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[7]} 0 1
   [junit4]   2> 225816 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[8]} 0 1
   [junit4]   2> 225822 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[8]} 0 1
   [junit4]   2> 225827 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[9]} 0 1
   [junit4]   2> 225831 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[9]} 0 1
   [junit4]   2> 225837 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[10]} 0 1
   [junit4]   2> 225842 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[10]} 0 1
   [junit4]   2> 225849 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[11]} 0 1
   [junit4]   2> 225854 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[11]} 0 1
   [junit4]   2> 225859 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[12]} 0 1
   [junit4]   2> 225864 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[12]} 0 1
   [junit4]   2> 225869 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[13]} 0 1
   [junit4]   2> 225875 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[13]} 0 1
   [junit4]   2> 225882 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[14]} 0 1
   [junit4]   2> 225887 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[14]} 0 1
   [junit4]   2> 225891 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[15]} 0 1
   [junit4]   2> 225899 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[15]} 0 1
   [junit4]   2> 225915 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[16]} 0 1
   [junit4]   2> 225920 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[16]} 0 0
   [junit4]   2> 225926 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[17]} 0 1
   [junit4]   2> 225932 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[17]} 0 1
   [junit4]   2> 225937 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[18]} 0 1
   [junit4]   2> 225944 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[18]} 0 1
   [junit4]   2> 225948 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[19]} 0 1
   [junit4]   2> 225953 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[19]} 0 1
   [junit4]   2> 225958 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[20]} 0 1
   [junit4]   2> 225962 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[20]} 0 1
   [junit4]   2> 225967 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[21]} 0 1
   [junit4]   2> 225972 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[21]} 0 1
   [junit4]   2> 225978 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[22]} 0 1
   [junit4]   2> 225990 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[22]} 0 1
   [junit4]   2> 226005 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[23]} 0 5
   [junit4]   2> 226009 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[23]} 0 1
   [junit4]   2> 226015 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[24]} 0 1
   [junit4]   2> 226019 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[24]} 0 1
   [junit4]   2> 226024 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[25]} 0 2
   [junit4]   2> 226030 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[25]} 0 1
   [junit4]   2> 226036 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[26]} 0 1
   [junit4]   2> 226053 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[26]} 0 1
   [junit4]   2> 226058 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[27]} 0 1
   [junit4]   2> 226062 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[27]} 0 1
   [junit4]   2> 226068 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[28]} 0 1
   [junit4]   2> 226072 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[28]} 0 1
   [junit4]   2> 226078 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[29]} 0 1
   [junit4]   2> 226083 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[29]} 0 1
   [junit4]   2> 226088 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[30]} 0 1
   [junit4]   2> 226092 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[30]} 0 1
   [junit4]   2> 226097 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[31]} 0 1
   [junit4]   2> 226101 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[31]} 0 1
   [junit4]   2> 226105 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[32]} 0 1
   [junit4]   2> 226110 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[32]} 0 1
   [junit4]   2> 226115 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[33]} 0 1
   [junit4]   2> 226120 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[33]} 0 1
   [junit4]   2> 226127 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[34]} 0 1
   [junit4]   2> 226132 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[34]} 0 1
   [junit4]   2> 226136 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[35]} 0 1
   [junit4]   2> 226140 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[35]} 0 1
   [junit4]   2> 226145 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[36]} 0 1
   [junit4]   2> 226154 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[36]} 0 1
   [junit4]   2> 226167 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[37]} 0 1
   [junit4]   2> 226174 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[37]} 0 2
   [junit4]   2> 226179 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[38]} 0 1
   [junit4]   2> 226185 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[38]} 0 1
   [junit4]   2> 226195 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[39]} 0 1
   [junit4]   2> 226199 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[39]} 0 1
   [junit4]   2> 226204 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[40]} 0 1
   [junit4]   2> 226218 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[40]} 0 1
   [junit4]   2> 226230 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[41]} 0 1
   [junit4]   2> 226236 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[41]} 0 1
   [junit4]   2> 226242 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[42]} 0 1
   [junit4]   2> 226247 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[42]} 0 1
   [junit4]   2> 226252 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[43]} 0 1
   [junit4]   2> 226259 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[43]} 0 1
   [junit4]   2> 226265 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[44]} 0 2
   [junit4]   2> 226270 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[44]} 0 1
   [junit4]   2> 226280 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[45]} 0 5
   [junit4]   2> 226289 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[45]} 0 1
   [junit4]   2> 226297 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[46]} 0 1
   [junit4]   2> 226302 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[46]} 0 1
   [junit4]   2> 226306 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[47]} 0 1
   [junit4]   2> 226318 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[47]} 0 1
   [junit4]   2> 226325 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[48]} 0 1
   [junit4]   2> 226331 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[48]} 0 1
   [junit4]   2> 226335 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[49]} 0 0
   [junit4]   2> 226342 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[49]} 0 1
   [junit4]   2> 226348 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[50]} 0 1
   [junit4]   2> 226354 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[50]} 0 1
   [junit4]   2> 226359 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[51]} 0 1
   [junit4]   2> 226364 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[51]} 0 1
   [junit4]   2> 226384 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[52]} 0 8
   [junit4]   2> 226400 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[52]} 0 2
   [junit4]   2> 226406 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[53]} 0 1
   [junit4]   2> 226413 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[53]} 0 1
   [junit4]   2> 226419 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=id:53} 0 0
   [junit4]   2> 226423 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=id:53} 0 0
   [junit4]   2> 226435 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[54]} 0 1
   [junit4]   2> 226440 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[54]} 0 1
   [junit4]   2> 226445 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[55]} 0 1
   [junit4]   2> 226452 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[55]} 0 1
   [junit4]   2> 226457 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[56]} 0 1
   [junit4]   2> 226463 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[56]} 0 1
   [junit4]   2> 226466 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=id:56} 0 0
   [junit4]   2> 226469 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=id:56} 0 0
   [junit4]   2> 226476 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[57]} 0 1
   [junit4]   2> 226481 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[57]} 0 1
   [junit4]   2> 226488 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[58]} 0 1
   [junit4]   2> 226504 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[58]} 0 1
   [junit4]   2> 226515 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[59]} 0 1
   [junit4]   2> 226522 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[59]} 0 2
   [junit4]   2> 226536 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[60]} 0 1
   [junit4]   2> 226542 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[60]} 0 2
   [junit4]   2> 226552 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[61]} 0 2
   [junit4]   2> 226558 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[61]} 0 2
   [junit4]   2> 226564 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[62]} 0 1
   [junit4]   2> 226583 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[62]} 0 1
   [junit4]   2> 226591 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[63]} 0 1
   [junit4]   2> 226599 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[63]} 0 1
   [junit4]   2> 226604 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[64]} 0 2
   [junit4]   2> 226610 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[64]} 0 1
   [junit4]   2> 226617 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[65]} 0 1
   [junit4]   2> 226624 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[65]} 0 2
   [junit4]   2> 226629 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[66]} 0 2
   [junit4]   2> 226634 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[66]} 0 1
   [junit4]   2> 226640 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[67]} 0 1
   [junit4]   2> 226646 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[67]} 0 1
   [junit4]   2> 226651 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[68]} 0 1
   [junit4]   2> 226657 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[68]} 0 1
   [junit4]   2> 226663 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[69]} 0 1
   [junit4]   2> 226668 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[69]} 0 1
   [junit4]   2> 226676 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[70]} 0 1
   [junit4]   2> 226681 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[70]} 0 1
   [junit4]   2> 226689 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[71]} 0 1
   [junit4]   2> 226694 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[71]} 0 1
   [junit4]   2> 226702 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[72]} 0 1
   [junit4]   2> 226710 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[72]} 0 1
   [junit4]   2> 226716 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[73]} 0 1
   [junit4]   2> 226722 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[73]} 0 2
   [junit4]   2> 226727 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[74]} 0 1
   [junit4]   2> 226734 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[74]} 0 1
   [junit4]   2> 226742 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[75]} 0 1
   [junit4]   2> 226749 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[75]} 0 2
   [junit4]   2> 226755 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[76]} 0 1
   [junit4]   2> 226763 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[76]} 0 1
   [junit4]   2> 226770 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[77]} 0 1
   [junit4]   2> 226776 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[77]} 0 1
   [junit4]   2> 226785 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[78]} 0 1
   [junit4]   2> 226791 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[78]} 0 1
   [junit4]   2> 226800 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[79]} 0 1
   [junit4]   2> 226805 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[79]} 0 1
   [junit4]   2> 226810 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[80]} 0 1
   [junit4]   2> 226819 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[80]} 0 1
   [junit4]   2> 226827 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[81]} 0 1
   [junit4]   2> 226834 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[81]} 0 1
   [junit4]   2> 226843 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[82]} 0 1
   [junit4]   2> 226849 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[82]} 0 1
   [junit4]   2> 226853 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[83]} 0 1
   [junit4]   2> 226857 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[83]} 0 1
   [junit4]   2> 226863 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[84]} 0 2
   [junit4]   2> 226873 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[84]} 0 1
   [junit4]   2> 226884 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[85]} 0 1
   [junit4]   2> 226889 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[85]} 0 1
   [junit4]   2> 226894 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[86]} 0 1
   [junit4]   2> 226898 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[86]} 0 1
   [junit4]   2> 226903 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[87]} 0 1
   [junit4]   2> 226907 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[87]} 0 1
   [junit4]   2> 226911 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=id:87} 0 0
   [junit4]   2> 226915 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {deleteByQuery=id:87} 0 0
   [junit4]   2> 226920 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[88]} 0 1
   [junit4]   2> 226924 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[88]} 0 1
   [junit4]   2> 226929 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[89]} 0 1
   [junit4]   2> 226934 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[89]} 0 1
   [junit4]   2> 226940 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[90]} 0 1
   [junit4]   2> 226948 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[90]} 0 3
   [junit4]   2> 226953 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[91]} 0 1
   [junit4]   2> 226958 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[91]} 0 1
   [junit4]   2> 226963 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[92]} 0 1
   [junit4]   2> 226973 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[92]} 0 1
   [junit4]   2> 226980 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[93]} 0 4
   [junit4]   2> 226985 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[93]} 0 1
   [junit4]   2> 226991 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[94]} 0 1
   [junit4]   2> 227008 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[94]} 0 1
   [junit4]   2> 227013 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[95]} 0 1
   [junit4]   2> 227017 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[95]} 0 1
   [junit4]   2> 227023 INFO  (qtp483398844-798) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[96]} 0 1
   [junit4]   2> 227028 INFO  (qtp83200478-816) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[96]} 0 1
   [junit4]   2> 227032 INFO  (qtp483398844-799) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[97]} 0 1
   [junit4]   2> 227037 INFO  (qtp83200478-817) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[97]} 0 1
   [junit4]   2> 227042 INFO  (qtp483398844-800) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[98]} 0 2
   [junit4]   2> 227046 INFO  (qtp83200478-818) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[98]} 0 0
   [junit4]   2> 227050 INFO  (qtp483398844-801) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[99]} 0 1
   [junit4]   2> 227054 INFO  (qtp83200478-819) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={wt=javabin&version=2} {add=[99]} 0 0
   [junit4]   2> 227057 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 227318 INFO  (qtp483398844-797) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@448f3ab0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17561bfa; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@448f3ab0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17561bfa; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_3,generation=3}
   [junit4]   2> 227318 INFO  (qtp483398844-797) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 227319 INFO  (qtp483398844-797) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@50da818a[collection1] main
   [junit4]   2> 227319 INFO  (searcherExecutor-204-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@50da818a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.3.0):C100/3:delGen=1)))}
   [junit4]   2> 227320 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 227320 INFO  (qtp483398844-797) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/b_s path=/update params={softCommit=false&waitSearcher=true&commit=true&wt=javabin&version=2} {commit=} 0 263
   [junit4]   2> 227323 INFO  (qtp83200478-815) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 227544 INFO  (qtp83200478-815) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@22e82fb1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39931f5e; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@22e82fb1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39931f5e; maxCacheMB=0.4326171875 maxMergeSizeMB=0.5859375)),segFN=segments_3,generation=3}
   [junit4]   2> 227544 INFO  (qtp83200478-815) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 227546 INFO  (qtp83200478-815) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2d2bfef8[collection1] main
   [junit4]   2> 227547 INFO  (searcherExecutor-214-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collectio

[...truncated too long message...]

ave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150801_082319_834.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20150801_082319_834.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:733: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:524: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1449: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1003: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 347 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
ERROR: Failed to archive {README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J1-20150801_082319_833.events=solr/build/solr-core/test/temp/junit4-J1-20150801_082319_833.events, solr/build/solr-core/test/temp/junit4-J0-20150801_082319_833.events=solr/build/solr-core/test/temp/junit4-J0-20150801_082319_833.events, solr/build/solr-core/test/temp/junit4-J2-20150801_082319_834.events=solr/build/solr-core/test/temp/junit4-J2-20150801_082319_834.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
	at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
	at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
	at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:145)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException: unexpected EOF with 1024 bytes unread
	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:349)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 13 more
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 915 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/915/

2 tests failed.
REGRESSION:  org.apache.lucene.codecs.simpletext.TestSimpleTextTermVectorsFormat.testRamBytesUsed

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([4AF0D6F4E5896BB7]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.codecs.simpletext.TestSimpleTextTermVectorsFormat

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([4AF0D6F4E5896BB7]:0)




Build Log:
[...truncated 7029 lines...]
   [junit4] Suite: org.apache.lucene.codecs.simpletext.TestSimpleTextTermVectorsFormat
   [junit4]   2> jul 30, 2015 2:46:47 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.codecs.simpletext.TestSimpleTextTermVectorsFormat
   [junit4]   2>    1) Thread[id=147, name=TEST-TestSimpleTextTermVectorsFormat.testRamBytesUsed-seed#[4AF0D6F4E5896BB7], state=RUNNABLE, group=TGRP-TestSimpleTextTermVectorsFormat]
   [junit4]   2>         at org.apache.lucene.store.MockIndexInputWrapper.readByte(MockIndexInputWrapper.java:132)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextUtil.readLine(SimpleTextUtil.java:60)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextTermVectorsReader.readLine(SimpleTextTermVectorsReader.java:234)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextTermVectorsReader.get(SimpleTextTermVectorsReader.java:141)
   [junit4]   2>         at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:194)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:187)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:127)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4089)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
   [junit4]   2>         at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:260)
   [junit4]   2>         at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:74)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2>    3) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2>    4) Thread[id=146, name=SUITE-TestSimpleTextTermVectorsFormat-seed#[4AF0D6F4E5896BB7], state=RUNNABLE, group=TGRP-TestSimpleTextTermVectorsFormat]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:690)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:679)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextTermVectorsFormat -Dtests.method=testRamBytesUsed -Dtests.seed=4AF0D6F4E5896BB7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_GT -Dtests.timezone=America/Campo_Grande -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   7134s J0 | TestSimpleTextTermVectorsFormat.testRamBytesUsed <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4AF0D6F4E5896BB7]:0)
   [junit4]   2> jul 30, 2015 2:46:47 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> jul 30, 2015 2:47:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.simpletext.TestSimpleTextTermVectorsFormat: 
   [junit4]   2>    1) Thread[id=147, name=TEST-TestSimpleTextTermVectorsFormat.testRamBytesUsed-seed#[4AF0D6F4E5896BB7], state=RUNNABLE, group=TGRP-TestSimpleTextTermVectorsFormat]
   [junit4]   2>         at java.util.zip.CRC32.updateBytes(Native Method)
   [junit4]   2>         at java.util.zip.CRC32.update(CRC32.java:65)
   [junit4]   2>         at org.apache.lucene.store.BufferedChecksum.flush(BufferedChecksum.java:80)
   [junit4]   2>         at org.apache.lucene.store.BufferedChecksum.update(BufferedChecksum.java:47)
   [junit4]   2>         at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:42)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextUtil.readLine(SimpleTextUtil.java:60)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextTermVectorsReader.readIndex(SimpleTextTermVectorsReader.java:103)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextTermVectorsReader.<init>(SimpleTextTermVectorsReader.java:86)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextTermVectorsFormat.vectorsReader(SimpleTextTermVectorsFormat.java:40)
   [junit4]   2>         at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:120)
   [junit4]   2>         at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   2>         at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   2>         at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:617)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4020)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
   [junit4]   2>         at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:260)
   [junit4]   2>         at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:74)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> jul 30, 2015 2:47:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=147, name=TEST-TestSimpleTextTermVectorsFormat.testRamBytesUsed-seed#[4AF0D6F4E5896BB7], state=RUNNABLE, group=TGRP-TestSimpleTextTermVectorsFormat]
   [junit4]   2> jul 30, 2015 2:47:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=147, name=TEST-TestSimpleTextTermVectorsFormat.testRamBytesUsed-seed#[4AF0D6F4E5896BB7], state=RUNNABLE, group=TGRP-TestSimpleTextTermVectorsFormat]
   [junit4]   2>         at java.util.zip.CRC32.updateBytes(Native Method)
   [junit4]   2>         at java.util.zip.CRC32.update(CRC32.java:65)
   [junit4]   2>         at org.apache.lucene.store.BufferedChecksum.flush(BufferedChecksum.java:80)
   [junit4]   2>         at org.apache.lucene.store.BufferedChecksum.update(BufferedChecksum.java:59)
   [junit4]   2>         at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:145)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:142)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:127)
   [junit4]   2>         at org.apache.lucene.store.RateLimitedIndexOutput.writeByte(RateLimitedIndexOutput.java:66)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextTermVectorsWriter.write(SimpleTextTermVectorsWriter.java:190)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextTermVectorsWriter.addPosition(SimpleTextTermVectorsWriter.java:160)
   [junit4]   2>         at org.apache.lucene.codecs.TermVectorsWriter.addAllDocVectors(TermVectorsWriter.java:285)
   [junit4]   2>         at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:196)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:187)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:127)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4089)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
   [junit4]   2>         at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4731)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4757)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4748)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:260)
   [junit4]   2>         at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:74)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {pgcord=IB LL-L2, xylilgnkk=DFR I(F)B1, ygahwxjp=DFR I(F)L2, fkxrbgc=DFR I(ne)3(800.0), mjdgpvig=IB SPL-DZ(0.3), thmmcgxut=LM Jelinek-Mercer(0.100000), yrurbt=DFR I(ne)2, ykshdibjf=BM25(k1=1.2,b=0.75), ony=DFR I(ne)Z(0.3), jjcex=IB SPL-L3(800.0), gnzkiiui=DFR I(n)B1, pfromoq=DFR I(ne)B1, myg=DFR GL3(800.0), wi=IB LL-D3(800.0), xahbm=DFR GB1, tlpodetsal=IB SPL-L1, emrwhzhm=DFR G1, ybjio=DFR I(n)B2, bezusdj=LM Jelinek-Mercer(0.100000), jch=DFR I(F)L1, yaxgvq=IB SPL-D3(800.0), ceuyoa=DFR I(n)1, gylrhu=DFR GBZ(0.3), pclwr=DFR I(F)Z(0.3), uoxtwfwm=DFR I(n)L2, nkacoo=DFR I(F)B3(800.0), swlfh=IB SPL-L2, dndbtphjp=DFR I(F)LZ(0.3), pnmfdfcpqg=DFR I(ne)1, ijucwics=DFR G3(800.0), pxqfkyoie=DFR I(F)Z(0.3), rfcrfsvav=DFR I(ne)B3(800.0), aiksoj=IB SPL-D2, zohspiotc=DFR I(F)2, cuoa=DFR I(n)1, udakikw=DFR I(F)Z(0.3), bxve=DFR GL1, rtpdtdsvu=DFR I(n)B1, uecvjovy=IB LL-DZ(0.3), ernwmoe=IB LL-DZ(0.3), hjhenshms=DFR I(F)LZ(0.3), ydwlhi=DFR I(ne)BZ(0.3), yrytbkwz=DFR I(ne)2, pdxqum=DFR GLZ(0.3), hppecqy=DFR I(F)LZ(0.3), oeefbjck=DFR I(ne)3(800.0), jyjtv=DFR I(F)B3(800.0), cij=DFR GB2, mobtx=DFR I(ne)1, qhzy=DFR GB1, vecty=DFR I(ne)3(800.0), wlmirpgpj=DFR I(n)L1, iqyhbu=DFR GL2, qqsabuw=DFR I(ne)L3(800.0), nasrmaudi=DFR G1, yfdzu=DFR G2, ix=IB LL-L3(800.0), dmbqfrw=DFR I(F)2, drpsqnb=DFR I(F)1, jmbjqcbv=DFR I(ne)L1, zydib=DFR I(n)L1, qvmgq=DFR I(ne)L3(800.0), szwokrijz=DFR I(ne)B2, uqp=IB SPL-D1, vswymkoc=DFR I(ne)3(800.0), bdemnuzz=DFR GL1, gbtsicmqgl=DFR I(F)B1, vrvtqdckb=DFR I(ne)BZ(0.3), bweiwzrk=DFR I(n)L3(800.0), firhb=IB SPL-LZ(0.3), pak=DFR I(F)LZ(0.3), qyalfyj=IB SPL-DZ(0.3), wevi=DFR I(n)L3(800.0), knywluzxig=DFR GZ(0.3), mhn=LM Jelinek-Mercer(0.700000), habhcol=IB SPL-LZ(0.3), ogov=DFR I(F)3(800.0), eoxncadi=IB LL-D2, kmdqn=IB LL-L3(800.0), zwllgfob=DFR G3(800.0), jifsboyaw=DFR I(F)LZ(0.3), aig=DFR I(F)L1, lufcknvwm=IB LL-D2, pakdhlbzw=IB LL-D2, iwxeuf=DFR GB2, qvkfwr=LM Jelinek-Mercer(0.100000), ybzilvai=DFR I(n)L1, ukj=DFR I(ne)B1, axufmgb=DFR GB2, ikrujuya=DFR I(ne)1, ryscvdquma=IB LL-L3(800.0), pzo=DefaultSimilarity, usvsaiced=DFR I(n)BZ(0.3), rmbavwra=DFR I(F)LZ(0.3), xirpl=DFR GB3(800.0), tnwu=DFR I(n)B1, jxnieji=DFR I(n)BZ(0.3), ihpo=DFR GL2, yqtqzwuqxx=DFR GZ(0.3), ynbs=DFR GL2, vvgw=DFR I(F)Z(0.3), ffbfg=DFR I(n)Z(0.3), piqq=DFR I(ne)LZ(0.3), fhiqndhbep=DFR I(ne)LZ(0.3), ejk=DFR I(ne)Z(0.3), uowxugii=DFR I(n)L3(800.0), ouyw=DFR I(F)1, ffmnrsdo=IB LL-LZ(0.3), ehzggdf=IB LL-L3(800.0), ejud=DFR I(n)3(800.0), dgvqn=DFR I(ne)L2, mfd=IB LL-L3(800.0), hyjepzorqc=IB LL-LZ(0.3), gtwcai=DFR GBZ(0.3), cnxgcmtmaj=DFR GZ(0.3), jqylgmr=DFR I(F)BZ(0.3), tadjvub=DFR I(n)1, ybibcgmetn=DFR GB2, syitekqsst=DFR I(n)L1, hog=DFR I(n)1, phwqb=IB SPL-D1, yayxi=IB SPL-D3(800.0), uwblotigzt=IB SPL-D2, hesxsdk=DFR I(ne)L2, cgk=DFR GL3(800.0), qshrcn=DFR GL1, zchxasgm=IB LL-L2, kouse=IB LL-L3(800.0), mzwkluhxcv=DFR I(ne)BZ(0.3), kodtmo=IB SPL-L2, tx=DFR GB2, httfcwgzzs=DFR I(ne)L3(800.0), jjxrcn=DFR GLZ(0.3), npasoue=DFR GBZ(0.3), blu=DFR I(n)Z(0.3), qdxcpb=DFR I(n)2, znuwajjobi=DFR I(n)BZ(0.3), ucby=DFR I(n)L3(800.0), pljre=IB SPL-D2, nyajzecs=DFR I(F)B1, btjc=LM Jelinek-Mercer(0.700000), chhjjvn=DFR I(F)L1, bkb=DFR I(ne)3(800.0), aolfner=LM Jelinek-Mercer(0.100000), jaz=DFR I(n)Z(0.3), ebntquyiq=DFR I(F)LZ(0.3), igfqrl=LM Jelinek-Mercer(0.100000), dmweh=DFR I(n)L3(800.0), xkr=DFR I(F)2, ryjva=IB LL-D3(800.0), xdrrwhso=DFR I(ne)LZ(0.3), cepuuz=DFR I(ne)L2, eou=DFR GLZ(0.3), vkzbikdxz=DFR GB3(800.0), pluxxatsfo=DFR I(n)1, tlxmqqo=DFR I(ne)Z(0.3), edtimcol=IB LL-L2, nkl=DFR I(F)2, eydqiufcy=DFR I(n)B3(800.0), kyclhb=DFR I(ne)BZ(0.3), uqxnzyx=DFR I(n)L3(800.0), rs=IB LL-L1, ileum=IB SPL-L1, zzb=DFR G2, ufn=DFR I(ne)B3(800.0), pkesp=DFR I(F)L1, getksls=DFR I(F)2, drbilbq=DFR I(ne)L3(800.0), hfbne=DFR GL3(800.0), dtgindpw=LM Jelinek-Mercer(0.700000), sthw=DFR I(ne)2, vlr=DFR I(n)1, zwo=IB SPL-D3(800.0), tlzbr=IB SPL-L1, eh=DFR I(F)2, vsbwu=DFR I(n)L2, lmwsidm=IB LL-D2, tvjdlbpvi=DFR I(n)L2, iekbo=IB LL-D1, twrnrk=DFR I(n)3(800.0), yv=DFR I(n)BZ(0.3), zjmzltbui=IB SPL-LZ(0.3), yubs=DFR I(n)BZ(0.3), zkqqjwuw=DFR I(F)L2, mu=DFR GB2, hxmkz=IB LL-D1, ovzjbifirw=IB SPL-L1, nvydclc=DFR I(ne)B3(800.0), gdrcu=LM Jelinek-Mercer(0.700000), grtw=DFR I(ne)Z(0.3), rwt=DFR I(n)Z(0.3), rcovfacdeu=DFR I(ne)B1, hytcd=DFR I(F)1, bxc=IB LL-L1, cqofglwn=DFR I(ne)LZ(0.3), xhdpybxcms=IB SPL-L3(800.0), rvdm=DFR I(ne)L1, rbeknzmyhw=LM Jelinek-Mercer(0.100000), udzlm=DFR I(ne)L2, idr=DFR I(F)3(800.0), gyrqbpib=DFR I(n)1, vpyhbcs=DFR GB3(800.0), hti=DFR I(ne)L1, bmtgpermi=IB SPL-D2, bomnfazl=DFR GLZ(0.3), mloezezsm=DFR I(n)2, quivwxubs=IB SPL-D1, evzyi=DFR I(F)B3(800.0), keurgsran=IB LL-D1, th=DFR I(n)1, cofcvheim=DFR I(ne)L3(800.0), wyajxabt=DFR G2, pw=DFR I(F)2, gybzilc=DFR I(F)LZ(0.3), fmgwv=DFR I(n)Z(0.3), zalgn=DFR I(ne)L2, sniig=DFR I(ne)L2, tkgimlag=DFR G2, zabxiwdiv=IB SPL-L3(800.0), dvptor=DFR GL3(800.0), vrtb=BM25(k1=1.2,b=0.75), xvijnte=IB LL-D1, lrfwzzdzw=DFR I(n)L1, cgwhzqn=IB SPL-D3(800.0), rzdj=DFR GB1, xdfsfkb=IB LL-D1, qwpfsogvwr=DFR GZ(0.3), lbflqqpa=DFR I(ne)B2, qyrv=DFR I(ne)2, wbcaj=DFR I(ne)L1, gzmurc=IB SPL-D1, yibwyicynr=DFR I(F)2, swgksg=DFR I(F)1, pveo=DFR I(ne)1, xcrmu=DFR I(n)L2, cmgefgxs=DFR I(ne)B1, zksptfzihn=DFR I(F)3(800.0), hnoeoxp=DFR GBZ(0.3), ygud=DFR GB3(800.0), rxlodtnty=IB SPL-L3(800.0), qervgody=DFR I(F)L1, bjzmhetvs=DFR GBZ(0.3), buiwmuwb=DFR G1, prl=IB LL-D3(800.0), jq=LM Jelinek-Mercer(0.700000), zlububkfbx=DFR GL1, oovmsgvha=DFR I(n)LZ(0.3), rhwihdoym=DFR I(ne)L1, ilos=DFR GL3(800.0), wwgos=DFR I(F)3(800.0), bwm=DFR I(F)L2, pywinxatd=DFR I(n)3(800.0), zrrmvwngj=IB SPL-LZ(0.3), iypllffmp=DefaultSimilarity, aoeu=DFR I(F)B2, qayughpywn=DFR I(n)B2, epgvff=DFR I(F)BZ(0.3), xwxcpy=DFR I(F)B3(800.0), ggktken=DFR I(ne)B3(800.0), yceqie=DFR I(F)B2, vgze=DFR GB2, buqczndul=DFR I(F)BZ(0.3), nrqegcvw=IB LL-L2, gus=IB SPL-D2, ypm=DFR GB1, qsgvox=DFR I(ne)L3(800.0), pnlukd=DFR I(ne)L2, hkxzjdv=DFR I(n)L2, vb=DFR I(F)1, stsezi=IB LL-D3(800.0), aues=DFR I(ne)L3(800.0), upistpnax=IB LL-D1, iihi=DFR GB3(800.0), luckmatek=DFR I(F)B3(800.0), jjxyuahjq=IB SPL-L2, xqlxqrzw=DFR I(ne)3(800.0), cdgbqlo=DFR G1, tydnh=DFR I(n)BZ(0.3), ty=LM Jelinek-Mercer(0.100000), ogwzdz=DFR I(n)B3(800.0), xynqud=IB SPL-D2, isrzzrazhc=DFR I(ne)L3(800.0), av=DFR I(ne)3(800.0), muniyqpaae=DFR I(n)3(800.0), ucksvkft=DFR I(n)B3(800.0), jfv=IB SPL-D3(800.0), dn=DFR GB1, xls=DFR I(n)B2, dfcrwx=DFR GZ(0.3), mhrc=IB LL-D1, kdk=BM25(k1=1.2,b=0.75), sgfecbvfiq=DFR I(F)LZ(0.3), evkqqnphy=DFR I(n)BZ(0.3), jmmpdjoh=IB SPL-D2, ss=IB SPL-D1, gwutc=IB SPL-D1, tlndukxty=DFR I(n)L3(800.0), klucwzlg=DFR GBZ(0.3), aycmrnd=DFR I(F)L3(800.0), qqgknyezue=DFR I(n)LZ(0.3), mjiyagrv=DFR I(ne)LZ(0.3), peo=DFR I(ne)Z(0.3), fvjgiy=DFR GBZ(0.3), ezqcoowew=DFR GL2, sqehilvfb=DFR I(ne)L3(800.0), ghj=DFR I(n)L3(800.0), yiycimr=DFR I(n)3(800.0), ccdpqx=DFR G3(800.0), ydmb=DFR I(n)2, cswn=DFR I(n)B2, xuqybfji=DFR GB2, dpbwuhqfr=IB LL-LZ(0.3), uegmaaemzx=DFR I(n)3(800.0), ssifhxehy=DFR I(F)3(800.0), zv=DFR I(n)2, se=IB SPL-D3(800.0), vd=DFR I(F)LZ(0.3), ocuutp=DFR I(ne)1, xcokmc=DFR I(n)L2, jxivoj=DFR GZ(0.3), yvvshdo=DFR I(n)B3(800.0), kzxoa=IB LL-L2, eo=DFR GZ(0.3), zum=DFR I(n)L1, trteu=IB SPL-L2, rncdpkuz=IB LL-L2, onuervstxc=IB LL-D2, fvrploh=DFR I(n)L3(800.0), rkmnsp=DFR I(ne)3(800.0), szogsumdya=IB SPL-DZ(0.3), acgpgzdtdm=DFR I(F)B3(800.0), azyxegdxey=DFR GB1, fjhektax=DFR GB3(800.0), ol=IB LL-L1, kpci=DFR GLZ(0.3), ekekyypxi=DFR I(n)B3(800.0), qmiwlhzg=IB LL-L2, erdtrd=DFR I(ne)L1, kxh=DFR GBZ(0.3), loh=IB LL-D2, lom=BM25(k1=1.2,b=0.75), ehta=DFR I(ne)1, ygz=DFR G1, utfixmkrs=IB LL-L3(800.0), wgnwbgm=DFR I(F)B1, jehkjfc=DFR I(n)L2, wgx=DFR I(ne)B1, pycc=DFR GL1, pvhdkp=DFR I(n)L3(800.0), ujyuwv=IB LL-D1, stgwe=DFR I(F)1, solknxjbd=DefaultSimilarity, quwu=DFR I(F)BZ(0.3), eaazxx=DFR I(n)B2, wvdopava=IB SPL-L2, zfjtclekjp=DFR I(F)L1, zldnuphu=DFR I(ne)B1, mkh=DFR I(F)B2, anzg=IB SPL-L3(800.0), algxvsr=DFR GLZ(0.3), pe=DFR I(ne)LZ(0.3), aerdjxxshg=IB SPL-L1, gsuqm=IB LL-D2, ii=DFR GB2, ztvc=IB LL-L2, xwcqni=IB SPL-D1, idxxa=DFR I(n)L1, qdlzv=LM Jelinek-Mercer(0.100000), lfouwv=IB LL-D2, vrxhkiqiqp=DFR GBZ(0.3), ktg=IB SPL-L1, merzjmdgap=DFR I(F)3(800.0), xejaqwn=DFR G1, edkwoftbm=DFR G3(800.0), wfvvox=DFR I(n)3(800.0), wefjobfw=IB LL-D3(800.0), jcdcvml=DFR GBZ(0.3), gob=DFR I(F)L3(800.0), annewnxdc=DFR I(n)B1, tjxnygniik=DFR GL3(800.0), lretrpe=DFR GB1, ffb=DFR I(F)LZ(0.3), mulyga=IB LL-LZ(0.3), uywa=IB SPL-L3(800.0), kq=IB SPL-D3(800.0), mziueerv=DFR I(F)B3(800.0), ppvnwcgrhb=DFR G3(800.0), gullex=DFR I(F)L1, vnc=IB LL-DZ(0.3), hdaneaken=DFR I(F)B3(800.0), ptfkzac=DFR I(ne)LZ(0.3), ra=DFR I(n)L2, qoty=DFR GBZ(0.3), ltxhvef=DFR I(F)L1, xrotceio=DFR I(F)3(800.0), icwkepqi=DFR I(F)L1, vokhxafccb=DFR I(ne)L1, mwhoh=IB LL-L3(800.0), qzbj=DFR G3(800.0), zy=DFR I(F)B1, kfxsno=DFR I(n)L3(800.0), lfafnb=IB SPL-LZ(0.3), vgpsibpsu=DFR I(F)B3(800.0), yxwuvsvjp=IB SPL-D3(800.0), ezwlh=DFR I(F)L3(800.0), gngdkqjq=LM Jelinek-Mercer(0.100000), nhsw=DFR I(F)LZ(0.3), oaypmfjawh=DFR GZ(0.3), pkcq=IB LL-D2, lpcsimt=DFR I(ne)1, cmczvjl=DFR I(F)3(800.0), zgwb=DFR I(n)3(800.0), gwzuxngsdh=DFR GL3(800.0), qtneybifg=LM Jelinek-Mercer(0.100000), fmxwr=DFR GB2, yjtk=IB LL-D1, anfiuez=DFR GL2, futopki=DFR I(ne)L2, wjeit=DFR GL1, gpfmo=DFR I(ne)BZ(0.3), orx=IB LL-L1, emyligrn=DFR GB3(800.0), hnmvtt=IB SPL-LZ(0.3), ifsozltihq=DFR I(n)L2, mefv=IB SPL-D1, tyfkjjhq=DFR GL1, onyfvtn=DFR G1, fpqbbeolve=DFR I(F)B2, eqqopyfwm=DefaultSimilarity, ubjlbrq=IB SPL-L1, bdht=DFR I(F)L3(800.0), yggmqih=DFR I(n)LZ(0.3), ylauewece=DFR GB1, kfg=DFR I(n)B1, oufatyml=DFR GL3(800.0), bhqbnqxwpr=IB LL-LZ(0.3), qwxnoe=DFR G2, ydrioaqc=IB SPL-L3(800.0), cpywj=DFR I(n)B3(800.0), mb=DFR I(F)B3(800.0), pn=DFR I(F)BZ(0.3), oonzeljh=DFR I(ne)B2, bdqxppew=DFR I(F)B1, jbbhwtp=DFR I(F)3(800.0), icnnon=DFR GL3(800.0), idlcbboag=IB LL-DZ(0.3), jfi=DFR I(ne)L1, boxqga=DFR I(n)1, ljybxzlr=DFR GB3(800.0), uuu=DFR I(ne)LZ(0.3), tsux=DFR I(ne)Z(0.3), viahucrvqk=DFR I(F)LZ(0.3), kkqutthgh=DFR I(ne)BZ(0.3), ratmvl=DFR I(n)L1, ebh=DFR I(F)2, hbwss=IB SPL-D2, wmf=DFR I(n)3(800.0), qbc=DFR I(ne)L3(800.0), do=IB LL-L2, onxiwch=IB SPL-DZ(0.3), akz=IB LL-D1, sisz=IB SPL-D2, okswr=IB SPL-L1, agtgnhhsa=DFR I(ne)B3(800.0), mhzp=DFR GB3(800.0), oa=DFR I(F)B2, kiaotvubv=DFR GBZ(0.3), wzo=DFR I(n)B1, mmkgduziyo=DFR G2, bkj=IB LL-LZ(0.3), ugnrz=DFR I(F)L2, fotfe=DFR I(ne)Z(0.3), svwubku=DFR I(F)B3(800.0), zoycdcbuqd=DFR I(n)B1, otzxqy=IB LL-L1, qcleqmdcul=IB LL-L2, ddhcl=DFR I(F)B3(800.0), hgrjsc=DFR I(n)L3(800.0), emzusscmt=DFR I(ne)B3(800.0), iyo=DFR I(ne)BZ(0.3), crxzi=IB LL-D1, xztiex=DFR I(F)BZ(0.3), lxsrvromw=DFR I(F)LZ(0.3), ldb=IB SPL-D3(800.0), udzdshpzpj=IB SPL-L1, sk=IB LL-D1, xzkql=IB SPL-D1, ekypo=DFR I(F)B2, jskx=DFR I(F)B2, aqikdggkv=DFR I(ne)LZ(0.3), wjtpawfeu=DFR I(n)B2, wlllvaf=DFR GZ(0.3), hxpa=DFR I(n)BZ(0.3), iltp=IB LL-L2, xdnwpef=IB LL-D1, axlyt=DFR I(F)1, dgabrea=DFR I(F)LZ(0.3), rhbukc=DFR I(n)B3(800.0), hndmhiie=DFR I(F)Z(0.3), lojwzi=DFR G3(800.0), okzosrwuat=DFR I(ne)3(800.0), upodewyz=LM Jelinek-Mercer(0.700000), clhpp=DFR I(n)B2, ictzojem=DFR GB3(800.0), zcbkhppbr=IB SPL-D3(800.0), zhzupq=DFR GL2, enkag=DFR I(n)3(800.0), fykjvj=DFR I(ne)Z(0.3), fhqgaat=IB SPL-D1, gf=LM Jelinek-Mercer(0.700000), dlmboio=DFR I(F)Z(0.3), sdjdbnzxs=IB SPL-L2, vq=LM Jelinek-Mercer(0.700000), wq=IB SPL-D3(800.0), sbvwm=DFR G3(800.0), liehyy=DFR I(F)1, xfww=DFR G2, tkyxxuqfkg=DFR I(F)Z(0.3), vtsfka=DFR I(ne)3(800.0), ehamnoys=DFR I(F)B2, dshoj=DFR I(ne)2, btrvlud=DFR I(ne)B3(800.0), ivxjafni=DFR GB3(800.0), dsil=DFR G2, hsrbim=DefaultSimilarity, eppfgstjo=DFR I(ne)L2, dzqzaa=LM Jelinek-Mercer(0.100000), ckrnbjr=DFR GZ(0.3), qypr=BM25(k1=1.2,b=0.75), uzazwua=LM Jelinek-Mercer(0.100000), omwuuvokrt=IB LL-L2, pyqwsmm=DFR I(ne)L2, plo=DFR GLZ(0.3), dcdcm=DFR I(n)B2, glqq=DFR GB3(800.0), ghpz=DFR I(n)B2, ifvzdrooe=BM25(k1=1.2,b=0.75), pm=DFR I(n)BZ(0.3), ycbet=DFR I(F)L1, r=IB SPL-L2, yobi=LM Jelinek-Mercer(0.100000), joscsehteo=DFR I(ne)B3(800.0), f_OFFSETS=DFR GL3(800.0), qd=DFR GL3(800.0), msuya=IB LL-L3(800.0), dduymexxhb=DFR I(F)LZ(0.3), hxetovif=DefaultSimilarity, ahbvjo=DFR I(F)L2, rzmjilt=DFR I(ne)B1, uppzbtrrxe=DFR I(n)2, qisgaj=DFR I(F)3(800.0), wtyjxqlecb=LM Jelinek-Mercer(0.100000), dovimete=DFR I(ne)B2, bdady=DFR I(ne)1, ijagkfwc=DFR I(n)L1, vvmxqjm=DFR I(n)L3(800.0), vvza=DFR GBZ(0.3), avzpxevr=DFR GB2, fkt=DFR I(ne)B2, pbc=DFR I(F)B3(800.0), cjhkhjmbn=DFR I(n)Z(0.3), dy=DFR G3(800.0), lpu=IB LL-L1, jkbloom=DFR G2, cis=IB SPL-D2, yiwj=DFR I(ne)BZ(0.3), pmugomy=DFR I(n)1, olbhs=DFR I(ne)B1, ftehrx=DFR GB3(800.0), qhsvvqo=DFR I(ne)L3(800.0), vqz=DFR I(n)B2, ojnqxecxc=DefaultSimilarity, znuzouv=IB LL-L1, xjksmwfptg=DFR GLZ(0.3), xwaunp=DFR GL3(800.0), xirdu=DFR I(ne)Z(0.3), pt=IB LL-D1, pjotjono=IB LL-LZ(0.3), audbqkukvn=DFR I(F)BZ(0.3), laqlzy=IB LL-L1, oomad=DFR I(F)L2, lcmvtwille=DFR I(n)BZ(0.3), fike=DefaultSimilarity, csxyopejc=DFR I(F)L2, wra=DFR I(ne)Z(0.3), veaxszza=DFR I(ne)1, rep=DefaultSimilarity, suiv=DFR I(n)B2, ciqranpm=DFR GBZ(0.3), ntvqcrlaup=DFR I(ne)L2, yayjn=DFR GL1, aqaee=DFR G3(800.0), gnwwyn=DFR I(ne)2, xjue=BM25(k1=1.2,b=0.75), bcuvpyj=IB LL-LZ(0.3), lypgni=DFR GB3(800.0), niekhsrn=DFR I(n)1, getcljxutu=IB LL-D2, sxs=DFR I(F)BZ(0.3), m=DFR GL1, eovqfs=IB LL-D3(800.0), tvlvgo=IB SPL-D1, mywpdhp=DFR I(F)3(800.0), gaa=DFR GBZ(0.3), anqe=DFR I(ne)B3(800.0), sxq=IB SPL-D1, vlifnz=DFR GB3(800.0), setbeink=IB SPL-LZ(0.3), ameh=DFR GLZ(0.3), rdkvvfyr=IB SPL-D3(800.0), rkmxotgqgz=IB SPL-L1, nmfqs=DFR G3(800.0), blzatcohw=IB LL-D3(800.0), bvfq=DFR I(n)L2, twtb=DFR I(n)BZ(0.3), lkuakdi=IB LL-LZ(0.3), cwkru=DFR I(ne)Z(0.3), dulbqb=IB SPL-DZ(0.3), yod=DFR I(ne)Z(0.3), svjkliai=DFR GLZ(0.3), laj=LM Jelinek-Mercer(0.100000), ixussllnfr=DFR I(n)B1, sod=DFR I(F)Z(0.3), rqt=DFR I(ne)3(800.0), ba=IB LL-L3(800.0), uxiwtaz=DFR I(F)L2, mhzpkltpdm=DFR I(ne)LZ(0.3), hte=IB LL-L1, drnzczdy=DFR I(F)BZ(0.3), zpkotqij=IB LL-D1, rmcp=DFR I(ne)1, sfaldjaurc=DFR I(n)L3(800.0), sbfarjf=DFR I(F)L2, vmlmrpac=IB LL-D1, jrfezbe=IB LL-D3(800.0), srfkypeec=IB SPL-D1, lfoxvwm=DFR GB3(800.0), ejkymnxez=IB SPL-LZ(0.3), fr=IB SPL-DZ(0.3), cgqegf=DFR GL1, dtpqubrxl=DFR I(ne)B1, kpxnfmka=DFR I(F)Z(0.3), otvhurls=DFR I(ne)B3(800.0), mn=IB LL-D2, jb=DFR I(ne)1, jbdkx=DFR I(ne)3(800.0), cq=DFR GL2, trgvn=DFR I(ne)3(800.0), ocsdmfnmi=DFR I(F)1, fsjxsl=DFR I(n)B1, go=DFR I(n)2, dx=DFR I(n)2, ajdmgkzf=DFR GL2, qc=DFR I(n)L1, crdla=DFR I(F)B2, ovio=DFR GB3(800.0), zxbpboe=DFR I(ne)1, eimnrfro=IB SPL-L1, txmkcojgob=DFR I(F)BZ(0.3), bcrqup=DFR I(n)L3(800.0), gdmdgwfw=DFR I(n)BZ(0.3), npynwnx=DFR I(ne)L3(800.0), jihykoklxp=DFR GLZ(0.3), xjgxnicjfk=IB SPL-L2, ia=DFR I(n)LZ(0.3), mieewtvpnp=DefaultSimilarity, gsfj=IB SPL-DZ(0.3), qeuxwdeo=IB SPL-L2, tfc=DefaultSimilarity, yq=DFR I(ne)B1, gepd=DFR GZ(0.3), opus=IB LL-L2, jbjyty=DFR I(n)3(800.0), dvyctdsppe=BM25(k1=1.2,b=0.75), jmi=DFR I(ne)L3(800.0), ewk=IB SPL-D2, txmomzfq=DFR I(F)BZ(0.3), mqtiyl=DFR I(F)L2, cp=IB SPL-LZ(0.3), jodytkz=DFR I(F)B1, ws=DFR I(n)3(800.0), exnghoshyr=DFR GBZ(0.3), kw=IB LL-D1, vvntqnv=DFR I(F)3(800.0), jtqykyejzq=DFR I(F)1, zdwrnw=IB LL-D3(800.0), ynw=IB SPL-L1, sd=LM Jelinek-Mercer(0.100000), vfindhotzs=DFR I(ne)B3(800.0), rlqtvo=DFR GB1, cqjegq=IB LL-D2, mddkw=LM Jelinek-Mercer(0.100000), yncwk=DFR I(ne)Z(0.3), royzb=DFR I(n)1, eoovuefjp=DFR I(F)3(800.0), =IB SPL-D1, ssyjdppu=DFR I(ne)Z(0.3), zlw=DFR GB3(800.0), ck=DFR I(n)B1, bxvx=DFR I(n)LZ(0.3), rmrali=IB LL-LZ(0.3), lmdpyxrsgp=IB SPL-L2, biwf=DFR GB1, ajdqx=DFR I(F)L3(800.0), oma=DFR I(F)2, uzqmuattk=DFR I(ne)Z(0.3), uqky=IB SPL-L2, ipiwvsfzx=DFR G2, mmlvpbkf=DFR I(n)Z(0.3), vdoj=LM Jelinek-Mercer(0.100000), ky=IB SPL-L2, syytidsgb=DFR GL2, djsxrikzfe=DFR I(F)L2, bfxl=DFR I(F)Z(0.3), xwzlbmclj=DFR I(n)B3(800.0), fncab=DFR I(ne)B1, crvxcrwus=DFR I(F)L1, yvadr=DFR I(F)L3(800.0), cmd=DFR I(ne)L3(800.0), lacffzo=DFR G3(800.0), yjgj=DFR I(n)1, jirlkybabq=DFR I(n)L3(800.0), dwgfrxu=DFR I(ne)3(800.0), axoovqccfg=DFR GB3(800.0), teuuazrzm=DFR GL3(800.0), idqxgopqyk=DFR I(ne)L1, obxvvc=DFR I(F)BZ(0.3), klp=DFR I(F)2, pbusp=DFR I(ne)L2, xnokppy=DFR I(n)L3(800.0), lzrmqtx=DFR I(n)1, nppapqut=DefaultSimilarity, ykuobgyg=DFR I(ne)BZ(0.3), vlnxil=LM Jelinek-Mercer(0.100000), rh=IB SPL-L3(800.0), qzf=DFR I(F)B3(800.0), cola=IB SPL-D2, efxdkn=DFR G1, trqyle=DFR I(F)B3(800.0), rzfgphrexq=LM Jelinek-Mercer(0.700000), hekivhzd=DFR I(ne)B3(800
   [junit4]   2> .0), xqtxavcdna=DFR I(F)L3(800.0), ktxuynmu=DFR GLZ(0.3), rblmfc=DFR I(F)LZ(0.3), jhnz=DFR I(F)LZ(0.3), pb=DFR GB1, soh=DFR I(F)L2, lngwjhe=DefaultSimilarity, ydvr=DFR I(n)Z(0.3), kfvrtdk=DFR I(F)1, rpsoo=IB SPL-LZ(0.3), sfhmpi=DFR I(F)2, ydr=DFR I(n)L2, npgtksqh=DFR GB1, kg=DFR I(n)LZ(0.3), tggsluuf=DFR G3(800.0), jsye=DFR I(F)B3(800.0), dvlp=IB LL-D3(800.0), cazadrpp=DFR I(ne)BZ(0.3), lfkxpo=IB SPL-L2, ehhax=DFR I(ne)1, qnscgpxmwn=IB SPL-D3(800.0), vswtogwae=IB LL-L1, ijaeetwlx=DFR GL1, zorxg=DFR G1, echl=DFR I(ne)L3(800.0), zgiqmqprb=DFR I(ne)LZ(0.3), wvvbtk=IB SPL-L3(800.0), ybjoteyr=IB SPL-DZ(0.3), gwj=DFR G2, zyknmmmx=DFR GB3(800.0), f_POSITIONS=DFR I(ne)3(800.0), uynkcwgi=DFR I(ne)BZ(0.3), xjqult=IB SPL-L3(800.0), ohzhfoe=IB LL-L2, sgvgxipe=DFR GL1, pouhrutv=DFR I(F)3(800.0), uvbnryidk=DFR I(n)1, qwyunhn=DFR I(ne)L3(800.0), valbgk=IB LL-DZ(0.3), ckrhwfmxx=DFR I(n)3(800.0), dwybji=DFR G3(800.0), rklblrc=DFR I(F)LZ(0.3), uy=DFR I(F)L2, tynb=IB SPL-LZ(0.3), bkg=IB LL-D3(800.0), kkzcwet=DFR I(ne)B2, ffpokdbyf=DFR I(n)Z(0.3), ewumlnh=DFR I(ne)2, lugkigbdt=DFR I(F)2, yxvecyq=DFR I(ne)BZ(0.3), blmd=IB LL-D2, trpdatv=DFR GBZ(0.3), nivdf=IB SPL-LZ(0.3), qymg=DFR I(F)LZ(0.3), gwwhwttw=DFR I(F)B3(800.0), i=DefaultSimilarity, pynbxjeixj=DFR I(F)B1, ctbcfe=DFR I(n)L2, wzsczdmsan=DFR I(F)Z(0.3), eyiivhgnqd=DFR I(n)3(800.0), lwiqgpkcr=DFR I(F)B2, vp=IB SPL-L1, kme=IB LL-DZ(0.3), ffhyxcu=DFR GBZ(0.3), wccijljhvb=DFR GB1, lvsh=DFR GLZ(0.3), jziydpw=IB LL-D2, heml=DFR G1, muuwdcyf=DFR I(ne)L3(800.0), qioxsff=DFR GL2, vylizp=DFR I(F)LZ(0.3), aswxi=DFR I(F)LZ(0.3), mqvmu=DFR G2, qslwuen=IB SPL-D2, spuvga=IB SPL-DZ(0.3), pdghvoodj=DFR I(n)L2, bda=DFR I(n)B2, kmwbuuunda=IB LL-D2, jrmsecrrr=DFR I(ne)Z(0.3), kzxq=DFR I(ne)B1, hlotimsxzj=IB SPL-L1, cncc=IB LL-L2, dtss=DFR I(n)LZ(0.3), rdilc=IB SPL-L3(800.0), fqfsompur=IB LL-LZ(0.3), xdshte=DFR I(F)LZ(0.3), udx=DefaultSimilarity, kgmgs=DFR GB3(800.0), yzur=DFR GL2, sdrm=DFR I(n)B1, cubrlfxjg=IB LL-DZ(0.3), zhwoouaceo=DFR I(ne)1, bcknl=IB SPL-D1, lp=DFR I(n)2, dsdzv=DFR I(ne)LZ(0.3), zuyva=DFR G3(800.0), yb=DFR I(F)B3(800.0), pkczincyib=IB LL-D3(800.0), ixawwauexk=IB SPL-L2, avaszwlk=DFR I(ne)B2, sakc=DFR I(F)L3(800.0), rrzhqzi=DFR I(F)B2, kjaxf=DFR I(F)Z(0.3), papsusiz=IB LL-DZ(0.3), fjsgvvmzv=DFR GL3(800.0), ukddofmeh=DFR I(F)B3(800.0), wnocvp=DFR GLZ(0.3), aa=DFR G1, ziwdny=IB SPL-D2, kxrlfffcbt=IB SPL-L1, nsottwsyh=DFR I(n)BZ(0.3), waxwmpulby=IB SPL-D3(800.0), yc=BM25(k1=1.2,b=0.75), twkjg=DFR GL1, stuoudgf=DFR I(ne)B3(800.0), puhgsn=IB LL-LZ(0.3), lzhqiahztj=DFR I(n)BZ(0.3), rzv=DFR G2, jbowjx=IB SPL-L2, fspqlya=IB LL-D3(800.0), yyzi=DFR GL3(800.0), bmxkpny=DefaultSimilarity, fpzs=DFR GLZ(0.3), ncjldvbt=IB LL-L1, nqdhcfij=LM Jelinek-Mercer(0.100000), hfrq=IB LL-D2, bmnksdbdt=DFR I(ne)3(800.0), kmsnxnt=DFR I(F)3(800.0), womkva=DFR I(ne)B2, sdp=DFR I(ne)B2, snwhkhmjr=DFR I(ne)B1, lpbtxpqrxb=DFR G3(800.0), xv=IB SPL-L1, iwcctxtpcu=DFR I(n)2, suuw=DFR G1, kif=DFR I(ne)B1, htgakcgnt=DFR I(F)2, zpfmrmllxw=DFR G3(800.0), cgrwctna=DFR G3(800.0), nby=DFR G3(800.0), yskg=DFR I(F)LZ(0.3), ykftpv=DFR GB1, owsozcfj=IB LL-D2, kteozx=DFR I(n)L2, prfh=DFR I(ne)2, gzabgoj=DFR GL2, ynxgatifo=DFR I(n)L2, oiyuq=DFR I(n)L2, nlm=DFR I(n)B2, gwudo=DFR I(ne)1, zadjv=DFR I(n)L3(800.0), cpady=LM Jelinek-Mercer(0.100000), zjpyjpgzup=DFR I(ne)BZ(0.3), lylvuagcf=DFR I(ne)3(800.0), lkcfrwwmh=DFR I(ne)B2, kobyolqksw=DFR I(F)L3(800.0), hjqczuew=IB LL-DZ(0.3), jdsqhm=DFR GLZ(0.3), syv=DFR GB2, eyfqyadyc=DFR I(n)B3(800.0), ylrqvdqqgf=DFR I(F)3(800.0), huskluc=DFR I(F)B3(800.0), uly=DFR I(F)B2, enz=IB LL-L1, kkphgl=DFR I(ne)B2, iv=DFR I(F)B3(800.0), bmxm=DFR I(n)2, xa=DFR I(n)BZ(0.3), ipwif=DFR I(n)BZ(0.3), lre=IB LL-DZ(0.3), pbou=IB SPL-D1, gh=DFR I(ne)L1, wnw=DFR GB2, mkpi=DFR GL3(800.0), qm=DFR GB2, krdtmzl=DFR I(n)B1, wcshmwof=DFR G2, hlac=BM25(k1=1.2,b=0.75), et=DFR I(ne)B2, icwdqm=LM Jelinek-Mercer(0.100000), yqbmuae=IB SPL-L1, roozcwvvdf=DFR GZ(0.3), iiikg=DFR I(n)BZ(0.3), bjwzjieubp=DFR I(ne)2, fldqzblbk=IB SPL-L1, njj=DFR G2, nyzxmfgv=DFR G3(800.0), xk=DFR GBZ(0.3), jjhmijowaw=IB LL-D1, nrdua=DFR I(ne)2, boyxa=DFR GL1, zz=DFR GB2, yomvjsvjt=DFR I(F)L2, uumjrbcj=IB LL-D1, woypohifbj=LM Jelinek-Mercer(0.700000), wcsgn=IB SPL-L3(800.0), iatdimhbc=DFR I(ne)B2, zct=DFR I(n)B2, jlemhk=DFR I(F)2, lntwww=IB LL-DZ(0.3), jzpkwlfk=IB LL-L2, gt=IB LL-L1, dbrose=DFR GL3(800.0), uyvmsqsq=DFR I(n)Z(0.3), yfuhj=DFR I(ne)Z(0.3), mmdzfwrqx=DFR G2, ucwldyu=IB LL-L2, eidmg=IB SPL-L3(800.0), fowfgly=DFR GB2, tphyafjnt=DFR I(F)B2, ghznjhqid=DFR GBZ(0.3), ahnxga=LM Jelinek-Mercer(0.100000), dheucadwv=DFR I(ne)B2, wfbe=IB SPL-L3(800.0), bjenislxki=DFR I(F)B3(800.0), ee=IB LL-D1, svsv=DFR I(F)LZ(0.3), eecwkr=IB LL-L1, eyorflxww=DFR I(F)B1, wqdtzah=IB LL-L3(800.0), dohrjgs=DFR GB3(800.0), kixnhbzyjs=DFR I(ne)Z(0.3), bcij=DFR I(F)L3(800.0), hkkhq=DFR I(F)B3(800.0), itpyylos=DFR I(n)B3(800.0), ljik=DFR I(ne)L2, zdwmryiqyj=DFR I(ne)2, xze=IB LL-DZ(0.3), qgpygraq=DFR GB1, mhln=DFR I(ne)L1, vmuj=DFR I(ne)B1, eyvt=IB SPL-L1, uxuxzrsls=IB LL-D1, gjyvifrul=IB SPL-D1, jxw=IB LL-D2, xeizzfa=DFR GL3(800.0), ilcir=DFR I(F)B3(800.0), nghfniteb=IB SPL-LZ(0.3), wupabky=IB LL-LZ(0.3), woeukkn=DFR GL2, iqhz=IB SPL-L1, hccjmbq=DFR I(F)B2, ymi=DFR G3(800.0), xw=LM Jelinek-Mercer(0.700000), yf=LM Jelinek-Mercer(0.100000), blvnr=DFR I(ne)L2, yajwf=DFR I(ne)L1, hvgqifecc=DFR I(ne)B1, pzll=IB SPL-D3(800.0), tiaw=DFR I(n)Z(0.3), uxuqdfjpox=DFR I(n)2, nxmmmt=IB LL-DZ(0.3), ulnhxaqjue=DFR I(ne)BZ(0.3), izcdr=DFR I(ne)1, dpuydb=DFR GL2, psouz=IB LL-L1, uayrjnlxe=DFR I(ne)2, rgjgyk=DFR I(F)L2, hlwnm=IB SPL-D3(800.0), qrg=DFR I(n)L1, pdhc=DFR I(n)L2, pee=DFR I(n)B1, kpcpziwdm=IB SPL-D2, ikbgkup=DFR I(F)2, qew=DFR GBZ(0.3), xqkhag=IB SPL-DZ(0.3), kyfpwnozay=DFR G2, ey=DFR I(ne)B1, hbvk=IB LL-DZ(0.3), wetweaq=IB SPL-D1, szhxyouvzt=DFR I(F)L2, dnusollp=DFR GB3(800.0), jzkguki=IB LL-L1, ebcnl=DFR I(ne)1, htyqoelhel=IB LL-L3(800.0), mnykqwy=DFR I(n)LZ(0.3), odvmqpf=DFR I(ne)B3(800.0), xckxysk=DFR I(ne)3(800.0), uaoyhq=DFR I(ne)3(800.0), u=DFR I(F)B3(800.0), mfybesnm=DFR I(F)3(800.0), fs=DFR I(ne)B3(800.0), ahqdeh=DFR GBZ(0.3), mnongtbc=BM25(k1=1.2,b=0.75), emccxh=IB SPL-DZ(0.3), cqax=DFR I(n)BZ(0.3), vpjfdteu=IB SPL-L3(800.0), uncjubkcbd=DFR GLZ(0.3), bzjizb=DFR GL1, gpttsyp=DFR I(n)L3(800.0), npagkux=DFR G2, uhmb=DFR I(F)Z(0.3), lgitl=IB SPL-L3(800.0), rxexft=IB LL-LZ(0.3), xn=DFR GL2, lkebocwrh=LM Jelinek-Mercer(0.700000), etjy=DFR I(F)B2, bt=DFR I(ne)Z(0.3), wa=DFR GZ(0.3), buwpi=IB LL-LZ(0.3), iojskf=IB SPL-L3(800.0), sdgnn=IB LL-D3(800.0), zhkqhlazbh=DFR I(n)B2, agj=DFR I(n)B1, ubxn=IB LL-DZ(0.3), wywjfdl=DFR I(ne)B3(800.0), af=DFR I(ne)L1, hksdawaofs=DFR G3(800.0), twmj=DFR I(n)1, jky=DFR I(n)B1, aeykko=DFR G2, bcnelnnm=LM Jelinek-Mercer(0.700000), tqsppnq=DFR I(n)L2, st=DFR I(n)BZ(0.3), wjrxhltm=IB LL-D3(800.0), wlsylh=DFR GB3(800.0), ecywkwo=DFR I(F)B3(800.0), mtw=IB LL-D1, kaakpgd=DFR GLZ(0.3), iuiiqyc=LM Jelinek-Mercer(0.100000), gu=DFR GZ(0.3), emkmre=DFR I(n)B3(800.0), gb=DFR I(n)B3(800.0), vmbot=DFR I(F)1, htmsdtgthm=IB SPL-DZ(0.3), kjkfqtjep=DFR I(F)Z(0.3), znh=DFR I(ne)B1, xjmg=DFR I(n)LZ(0.3), fpbf=DFR GL2, wylw=DFR I(F)B3(800.0), xxjmwmy=DFR GB2, jzidpwv=DFR I(ne)B1, lllyuypkmd=DFR I(F)L1, neeuvl=LM Jelinek-Mercer(0.700000), yjzdxyj=DFR I(F)B2, odofkvex=DFR GZ(0.3), umcsnajshi=DFR I(ne)LZ(0.3), bdmu=DFR I(ne)L3(800.0), jg=DFR I(n)L3(800.0), plkdqpr=DFR I(ne)L3(800.0), hxclyezz=IB LL-L2, ezdwatak=DFR I(F)3(800.0), hwg=DFR I(F)Z(0.3), ukk=DFR I(n)L3(800.0), ccfgydf=IB SPL-D3(800.0), cnu=DFR G2, apy=DFR I(F)L3(800.0), gojbmnp=DFR GLZ(0.3), hvplgcrpj=DFR GZ(0.3), rj=DFR I(F)Z(0.3), bfufvh=IB SPL-LZ(0.3), fofditu=IB LL-D2, ep=DFR I(n)Z(0.3), powfcgifnb=DFR GLZ(0.3), whg=IB SPL-L3(800.0), ttlc=IB LL-DZ(0.3), lorblxco=DFR I(ne)LZ(0.3), iurr=DFR GB2, onuy=DFR GBZ(0.3), aprfma=DFR I(n)L3(800.0), bk=IB LL-D2, sjykuixze=DFR G2, htzq=DFR GL3(800.0), weiujpyotp=IB SPL-L1, zjfdabmfc=DFR I(n)Z(0.3), ebor=IB SPL-D1, uaexcask=DFR I(ne)3(800.0), jcpb=IB LL-L2, cqbo=IB LL-L2, vfiypc=DFR I(ne)B3(800.0), nazgyzucfp=IB LL-L2, kr=DFR I(F)3(800.0), awzda=IB LL-D2, zlqw=DFR I(ne)2, vv=IB LL-D1, ebevliaav=IB LL-D2, e=DFR G3(800.0), bulzpkatqm=DFR I(F)L1, dnoosbvfcv=DFR I(ne)2, svuawwgctt=DFR GL3(800.0), ltrsdarkv=IB LL-L1, mzvacxdmrr=DFR I(F)LZ(0.3), txdrvqgkx=DFR I(n)L2, nsvuaf=DFR I(F)3(800.0), dfco=IB SPL-L1, vwp=DFR I(F)L1, qoxqwlws=IB LL-D1, noeo=LM Jelinek-Mercer(0.700000), wokbjcazuo=LM Jelinek-Mercer(0.700000), ajmgjxqs=IB LL-L2, onayxvli=DFR I(F)B2, nphoqyhh=IB SPL-LZ(0.3), jkalbosw=DFR I(F)3(800.0), vc=DFR I(ne)LZ(0.3), wkujili=DFR GB3(800.0), ovliilcfkc=DFR I(F)L3(800.0), zekrdgbmsz=DFR GL1, pfg=DFR I(ne)3(800.0), xugrjtd=DFR I(F)L3(800.0), gvq=DFR GB1, pibzzksj=LM Jelinek-Mercer(0.700000), pruhofem=DFR G1, nndr=IB LL-D3(800.0), sauj=IB LL-L2, iina=IB LL-L2, yrrlbdb=DFR GBZ(0.3), urf=DFR I(ne)B3(800.0), iapcxtp=LM Jelinek-Mercer(0.100000), f_POSITIONS_AND_OFFSETS=DFR I(ne)Z(0.3), onyueoegnl=DFR I(n)B2, dghfgfaymq=DFR I(F)L2, oxdzgwrxn=DefaultSimilarity, cxpokzwe=DFR I(n)Z(0.3), lufuzq=DFR GL1, umkuq=DFR I(ne)B2, vgvcz=DFR I(ne)2, njg=DFR I(n)B2, uiadbfnk=DFR I(n)B1, jeefyrw=IB LL-D3(800.0), gta=DFR I(ne)B1, gwpu=DFR I(ne)3(800.0), bmjvmzhwv=DFR I(ne)L1, uofgjnzykd=IB LL-L2, elvix=BM25(k1=1.2,b=0.75), crfjoqm=IB LL-D1, nefznfasqh=DFR I(n)1, aumhpc=DFR I(F)B2, vltpcjjho=DFR I(ne)B1, ckrqaaqmzc=DFR GB2, earejtk=DFR GL2, bmkqp=IB LL-L2, trzrztbat=DFR I(ne)L1, nmwyxx=IB LL-L2, fbm=DFR GB3(800.0), zemqrcms=IB SPL-L1, ficrmof=DFR GB1, pdxvruqn=IB SPL-L1, focctbikq=DFR I(n)B1, mmwjrpo=IB SPL-L1, lmq=DFR I(F)1, oaj=DFR I(n)B1, mss=IB LL-D2, wquug=IB LL-LZ(0.3), xixdq=DFR GB2, qseyvtrxms=DFR I(n)2, nltm=IB LL-DZ(0.3), dnqqbyasi=DFR G2, xtqvzck=DFR I(F)B1, xkdzeiko=DFR I(F)B3(800.0), thxvsjty=DFR I(F)BZ(0.3), dhfryrbg=DFR I(n)L1, myt=IB SPL-DZ(0.3), wgaujqsyiu=IB SPL-D2, pijxt=DFR GL1, ehxeiztkr=IB SPL-L3(800.0), gzbwmizl=DFR I(F)BZ(0.3), ouylzdta=IB SPL-DZ(0.3), rjk=DFR I(F)L1, turnza=DFR I(F)Z(0.3), yojgeewuw=BM25(k1=1.2,b=0.75), lizpxfor=DFR I(F)L2, mwmmpq=IB LL-L1, iuvpj=DFR I(F)L2, kegpe=DFR G2, iapqeygfl=DFR I(F)Z(0.3), pmvfaiec=DFR I(ne)B3(800.0), efrs=DFR G2, otjjuae=IB SPL-D1, o=DFR GBZ(0.3), sjsxomw=DFR I(ne)B3(800.0), olkrirtjs=DFR I(ne)B1, mgbfurom=IB LL-L1, olose=DFR I(n)LZ(0.3), kmiuyle=DFR I(ne)B1, smoyudyg=DFR I(n)B2, uyd=IB SPL-D1, ogzk=IB SPL-LZ(0.3), dczsjmubg=DFR I(F)L1, zwcgqe=DFR I(F)3(800.0), peb=DFR I(ne)B3(800.0), lakxvv=IB SPL-L2, epyic=IB LL-D1, yg=IB SPL-D3(800.0), sfzn=DFR I(n)B3(800.0), iqmndjhkq=IB LL-L1, jobf=DFR I(n)BZ(0.3), uwlrwrrwd=DFR I(ne)1, yxszbiv=DFR I(n)B3(800.0), mjktifayec=DFR I(ne)B2, xetpdciop=IB SPL-L2, jjo=IB LL-L3(800.0), fhzwlx=LM Jelinek-Mercer(0.100000), lpwyjltk=DFR I(ne)LZ(0.3), brswmypc=IB LL-DZ(0.3), aowdiwwwwd=IB SPL-D3(800.0), qujfsliq=IB LL-L3(800.0), gv=DFR I(n)Z(0.3), yqsyln=DFR I(n)1, vaju=IB LL-D3(800.0), vhtbpwyfgq=DFR I(F)LZ(0.3), lvyjeg=DFR I(n)B3(800.0), fggvvswcp=IB LL-L1, qvjplnp=DFR I(F)L1, je=DFR GLZ(0.3), wdsdebhsaj=DFR I(n)L2, vkfrk=DFR G2, khz=LM Jelinek-Mercer(0.100000), xrx=DFR I(F)LZ(0.3), ztwyhmufp=DFR I(F)L3(800.0), zirncw=DFR I(n)L3(800.0), lcfwbt=DFR G2, mkpbe=DFR I(ne)LZ(0.3), lavpfmew=DFR I(F)1, lyxjxcyiws=IB SPL-L1, pwjwvdbc=IB LL-D2, wbasm=DFR I(n)3(800.0), pr=DFR I(F)B2, gl=IB SPL-D2, nbnqkjwqr=DFR GZ(0.3), xxxnhrjghi=DFR G2, bitjimvvgn=DFR I(ne)B1, chq=DFR I(F)Z(0.3), bs=DFR GB3(800.0), hw=DFR I(n)B3(800.0), ki=DFR GL2, ny=DFR I(n)B3(800.0), mrjepgm=DFR I(F)L3(800.0), tydyvei=DFR I(ne)B1, oftzwosg=DFR GLZ(0.3), wcar=DFR I(ne)LZ(0.3), yaxurqficb=DFR I(ne)Z(0.3), ihndv=DFR I(ne)L1, kxtfqgv=DFR I(n)L2, yvoxeqva=IB SPL-LZ(0.3), cern=IB LL-D3(800.0), ezzol=IB SPL-L2, tubew=DFR GBZ(0.3), fsovftvc=IB LL-LZ(0.3), uivakmvkk=IB SPL-D2, svtsjlz=DFR GZ(0.3), tjgev=BM25(k1=1.2,b=0.75), c=DFR I(ne)BZ(0.3), zutjlbn=DFR I(n)Z(0.3), kvn=DFR I(n)Z(0.3), sq=BM25(k1=1.2,b=0.75), vedjfb=DFR GL3(800.0), eenjbvc=DFR GL1, su=DFR I(F)BZ(0.3), fx=IB SPL-L3(800.0), qom=LM Jelinek-Mercer(0.700000), ir=IB SPL-D2, wayux=DFR I(F)L3(800.0), iviwacxw=IB SPL-DZ(0.3), meae=DFR I(n)1, rcxoktvlq=DFR I(F)LZ(0.3), feau=DFR I(n)B2, qhdy=DFR I(n)B1, twrwzxq=DFR G3(800.0), bn=DFR I(ne)B1, qdzi=DFR G2, effsadlpuu=DFR I(ne)B3(800.0), lx=DFR I(n)B1, dyenkq=DFR G1, veyfkdi=DFR G1, luipceua=DFR GZ(0.3), ghpgteayr=IB LL-L2, tntja=DFR I(n)BZ(0.3), paf=DefaultSimilarity, wikdgznd=DFR I(F)Z(0.3), lrvsheimji=DFR I(ne)B2, wmbh=DFR GB2, p=IB LL-D1, uadxcdxdef=DFR GL2, shlzs=DFR I(ne)2, axmuaygie=DFR GB2, srasebaxqq=LM Jelinek-Mercer(0.700000), dgijfnnkib=IB SPL-L2, enqiwipgr=DFR I(F)Z(0.3), mvrokpcbxj=IB SPL-D1, vgyavnvd=DFR I(F)B3(800.0), xwqmjxxxqj=DFR I(ne)Z(0.3), lywc=DFR I(n)L3(800.0), viskyo=DFR GBZ(0.3), afqqofh=IB SPL-DZ(0.3), ui=IB SPL-D1, hajnmengzf=DFR GZ(0.3), zspfu=BM25(k1=1.2,b=0.75), gcgmoyc=IB SPL-D2, pvtwyw=DFR I(ne)3(800.0), otmcpkkpq=IB SPL-L3(800.0), wzsmnexd=DFR I(ne)B1, mwezkaq=IB LL-D1, jaoafrrdik=DFR GL1, ciwn=DFR I(n)L3(800.0), sbfcbwzyuq=DFR I(n)3(800.0), jxhkoqw=IB SPL-DZ(0.3), au=DFR I(n)3(800.0), mje=DFR I(ne)1, evutohv=DFR I(n)L2, ayv=DFR I(ne)L3(800.0), zcww=DFR I(ne)L1, vcfeicjvkv=DFR I(F)2, ggno=DefaultSimilarity, dykenqbque=DFR GBZ(0.3), lm=IB SPL-LZ(0.3), jotnutn=DFR I(n)2, hjsuxsyl=DFR GL2, ewipovo=DFR I(n)L3(800.0), awnywg=DFR I(F)2, apzstnt=DFR I(F)BZ(0.3), lqpyfb=IB SPL-L1, fxpnktqhyj=DFR I(F)3(800.0), hsehw=DFR G3(800.0), jfzducgcl=DFR I(F)L2, jlwhagit=DFR I(ne)B1, hjrzetnjtt=DFR I(n)B2, evcrxzxy=DFR I(F)L1, ghvsytzpb=DFR I(n)L2, lkueflujdm=DFR I(n)L2, wlzx=DFR I(n)Z(0.3), axsgn=DFR I(F)L1, ccjxs=DFR I(ne)L1, kexfekp=DFR I(F)LZ(0.3), qyppuqo=DFR GBZ(0.3), fszhnepen=DFR I(n)L3(800.0), ohjvs=DFR I(F)B2, iqpat=DFR I(n)B3(800.0), hfsjzlc=IB SPL-DZ(0.3), yrcuhgmi=DFR I(F)L3(800.0), fuafr=IB SPL-L3(800.0), vtnx=DFR G1, omcugwca=DFR G2, xxrtzzklmc=DFR I(ne)LZ(0.3), rkbnpko=DFR I(F)3(800.0), evt=DFR I(n)Z(0.3), gjnfnlvb=DFR G3(800.0), wbhg=IB SPL-D3(800.0), kbz=DFR I(n)L2, ofxz=DFR I(F)B3(800.0), psshbiu=DFR I(n)L1, cyg=DFR I(n)BZ(0.3), cqgpqsop=IB SPL-L3(800.0), qujeqas=DFR I(F)L2, anbh=DFR I(n)B3(800.0), xx=DFR I(n)3(800.0), dhttjtrzl=IB LL-D3(800.0), uvvtvgelk=DFR I(ne)2, yfbtnqnqx=DFR I(F)B3(800.0), cipsg=IB SPL-D3(800.0), ngini=DFR I(n)Z(0.3), jpdwhosx=DFR I(n)B2, jiqlce=IB LL-D1, uphqutjvm=DFR I(F)BZ(0.3), dzoxaklg=DFR I(n)2, mtwvkabjzk=IB LL-DZ(0.3), socgnb=DFR I(ne)B1, ldjhkof=DFR I(n)B1, jnqnjsbqf=LM Jelinek-Mercer(0.100000), mpt=IB LL-L1, piehkg=DFR I(n)B1, hummbs=IB SPL-L1, rkewll=IB LL-L1, peitp=DFR I(n)2, uqubpry=DFR I(F)BZ(0.3), fula=DFR I(ne)B1, vktregw=DFR I(ne)Z(0.3), wrbjjco=DFR I(ne)BZ(0.3), yumjxaie=DFR GB1, jd=IB LL-D3(800.0), wpxznstxwp=IB SPL-LZ(0.3), uhvqxaqna=IB LL-LZ(0.3), hhzuq=IB LL-D3(800.0), ccan=DFR I(ne)LZ(0.3), prww=DFR GZ(0.3), way=IB LL-L3(800.0), dt=DFR I(n)LZ(0.3), nvoiubplhx=DFR G3(800.0), ljihfi=DFR I(F)Z(0.3), pvtmo=IB LL-D1, csvykj=IB LL-D3(800.0), koxvl=IB SPL-D3(800.0), qxi=DFR GB1, lqpieupba=DefaultSimilarity, rjndnkl=DFR GB3(800.0), ruwbqfwo=DFR I(n)Z(0.3), tde=IB LL-L1, pymbfr=DFR I(ne)Z(0.3), lykqoc=IB LL-D2, wtczea=DFR I(ne)L2, gieqscsph=DFR I(n)B3(800.0), bopfzuq=DFR I(F)3(800.0), bkz=IB SPL-LZ(0.3), kidplxwfq=DFR I(n)3(800.0), yhkvfget=IB SPL-L3(800.0), euo=DFR I(F)3(800.0), kfrn=DFR GL3(800.0), nnepp=IB LL-D2, okj=IB SPL-L2, mkvfgf=DFR I(n)B3(800.0), tqnfiayw=IB LL-DZ(0.3), lxduflcgt=DFR GLZ(0.3), leiiu=DFR I(ne)2, cm=IB SPL-L3(800.0), dqt=DFR I(n)L2, iexixbza=DFR I(n)L1, zlj=DFR I(ne)L2, mbs=IB LL-D3(800.0), jbjpcm=DFR I(n)3(800.0), ievaoiziqk=IB SPL-L2, dwvijknii=IB LL-DZ(0.3), jqtymhiq=DFR GL3(800.0), tlyruruq=DFR I(F)L3(800.0), yxp=DFR GB3(800.0), huanymac=IB LL-D1, xasmx=DFR I(ne)3(800.0), rfi=IB LL-D2, uobrdcn=DFR I(F)2, lwdl=DFR I(F)B1, mvwbg=DFR I(n)Z(0.3), jsrmr=IB SPL-L3(800.0), hi=DefaultSimilarity, zbfpt=DFR I(n)L1, ocnspq=BM25(k1=1.2,b=0.75), etnyk=DFR I(ne)LZ(0.3), msobkl=DFR I(F)B2, wvl=DFR GB3(800.0), muqxzu=DFR GZ(0.3), owskdjpo=DFR I(ne)B3(800.0), gax=DFR I(ne)L3(800.0), louogef=DFR I(n)B3(800.0), pcnjma=DFR I(n)BZ(0.3), czyivn=DFR G3(800.0), mabc=DefaultSimilarity, ergppat=DFR GBZ(0.3), lerxrjr=DFR I(ne)BZ(0.3), fjxz=IB SPL-D3(800.0), mfbrolz=DFR I(ne)2, kvfbj=DFR GL3(800.0), hole=DFR I(n)3(800.0), heselqxfgr=DefaultSimilarity, dp=DFR I(F)1, ayxzs=LM Jelinek-Mercer(0.100000), cjii=DFR GL3(800.0), qbtsux=DFR I(ne)L1, ychfonolg=DFR GL3(800.0), sz=IB LL-L2, fsso=DFR I(ne)3(800.0), azecqlh=DFR I(n)L1, yaavwazxi=DFR I(n)B1, qzbyyoxnv=IB SPL-L3(800.0), bfhhcxio=DFR I(ne)3(800.0), iunqawko=DFR GBZ(0.
   [junit4]   2> 3), vpllsxdll=DFR I(n)B2, suhhmcp=DFR G1, tjivtnqz=DFR GZ(0.3), tepkmlwab=DFR G3(800.0), bwctn=DFR I(ne)B1, mjns=DFR GB1, xrnpdsqvlo=DFR I(F)B1, tfpxn=DFR GB2, euqexxeoyl=DFR I(F)B1, qnve=IB SPL-D1, lpcvtco=IB LL-L1, gj=DFR I(ne)B2, kzzt=IB LL-D2, zs=IB SPL-LZ(0.3), efnizrrpl=DFR GL2, hnh=DFR I(n)L1, yofe=DFR I(ne)B3(800.0), qzc=IB SPL-L2, wytlekjljf=IB SPL-L1, ktxszwtsae=DFR I(ne)B1, cvh=IB SPL-LZ(0.3), xcxhfx=DFR I(n)L2, orzy=DFR I(ne)L3(800.0), wtpqvvnuz=DFR I(F)3(800.0), od=IB SPL-D2, pnzdldftcg=IB SPL-L3(800.0), qzigln=DFR I(F)3(800.0), yznpzkpeud=DFR GB2, kf=DFR GBZ(0.3), itfmolfkan=IB LL-L1, zogyhjx=DefaultSimilarity, sawoba=IB LL-L2, hb=BM25(k1=1.2,b=0.75), fpd=IB LL-D3(800.0), oxjbrytg=DFR I(ne)L1, ftebvr=IB SPL-LZ(0.3), ncpsghg=DFR I(ne)L3(800.0), hiopsv=DFR I(n)Z(0.3), xnklqvagge=DFR I(ne)2, fjaroapm=IB SPL-DZ(0.3), rhc=DFR G3(800.0), hdg=DFR GBZ(0.3), xkww=DFR GLZ(0.3), tfijhvdz=IB LL-LZ(0.3), apx=DFR I(n)L3(800.0), qorts=LM Jelinek-Mercer(0.700000), ogpu=DFR GB3(800.0), jx=DFR G2, bfqxladrx=DFR I(n)L1, uzvrtyi=IB LL-DZ(0.3), gbfexfqvme=DFR I(ne)LZ(0.3), lwucai=LM Jelinek-Mercer(0.700000), yiokmcnata=DFR GZ(0.3), theap=DFR I(n)B1, dxijhwwec=DFR I(ne)L1, trfbxcsd=IB SPL-L1, ela=DFR I(F)BZ(0.3), hyzrtxj=DFR I(n)BZ(0.3), uuki=DFR I(n)L1, kmxioj=DFR GL2, ngttcytrk=DFR G2, jy=IB LL-DZ(0.3), pxt=IB SPL-D1, evfxt=IB LL-D2, fm=IB LL-DZ(0.3), fskagxj=DFR I(ne)L3(800.0), jfc=IB LL-L3(800.0), oqomwgutk=IB SPL-DZ(0.3), fgxxedju=DFR I(ne)BZ(0.3), vtfclwcuz=BM25(k1=1.2,b=0.75), bsnnykurmi=DFR I(n)2, wklslhothm=DFR I(F)3(800.0), kqhmbzkt=DFR I(n)2, vjkmcs=DFR I(ne)L3(800.0), pa=DFR I(ne)B3(800.0), ttiuy=DefaultSimilarity, qj=DFR I(F)B3(800.0), qqnjynntk=IB LL-D1, hdqjl=IB LL-DZ(0.3), ajsbrd=DFR I(ne)Z(0.3), bw=DefaultSimilarity, xjmk=DFR I(ne)2, kzqfxtrsd=IB SPL-LZ(0.3), kp=LM Jelinek-Mercer(0.100000), kh=IB SPL-LZ(0.3), vzdfdjvuvi=BM25(k1=1.2,b=0.75), gtu=IB SPL-D1, zsskvbjdut=DFR I(ne)Z(0.3), jtcpyajbit=DFR I(ne)Z(0.3), vupfqrg=DFR I(ne)LZ(0.3), lwl=DFR I(n)BZ(0.3), xzyfhofacu=DFR GL3(800.0), jfrwjfer=DFR I(ne)L3(800.0), hkhrtompn=IB SPL-D2, mayoyxd=DFR GBZ(0.3), qzmdzr=DFR I(n)3(800.0), xvjyszy=IB LL-L3(800.0), fiojcx=DFR I(n)BZ(0.3), dutlbwpj=DFR I(n)2, sopffpkh=DFR I(n)L3(800.0), pdcsruz=IB SPL-D2, snaxdrg=DFR I(n)L3(800.0), dftdhtndt=DFR I(F)1, kqaw=DFR I(F)L2, iei=IB LL-D2, cuzycnsm=BM25(k1=1.2,b=0.75), dw=DFR I(ne)BZ(0.3), agzcwtnuts=DFR G2, jgr=IB SPL-LZ(0.3), jzus=DFR GL2, pcs=DFR I(n)LZ(0.3), vzyhxv=IB LL-L2, qkkcwbtv=DefaultSimilarity, vdkbmioegl=DFR I(ne)L2, amuitiqqb=DFR G3(800.0), grfcmoav=IB LL-L3(800.0), jorkxhbv=DFR I(F)BZ(0.3), kwifowquz=DFR I(F)B1, ahm=DFR I(n)L1, znieyk=DFR I(ne)L3(800.0), vilofntn=DFR I(ne)2, rdxdb=DFR I(n)1, ofvix=IB SPL-L3(800.0), ijef=DFR GZ(0.3), dnurf=IB LL-DZ(0.3), vxogy=DFR I(n)Z(0.3), flbq=DFR GB3(800.0), hweej=DFR I(n)B3(800.0), zfjis=IB SPL-DZ(0.3), ie=DFR I(n)2, ekqbupomd=IB LL-D1, bjenxns=DFR I(F)L2, vgailppwmb=DefaultSimilarity, zoglshbzlt=IB SPL-D2, jklnxeep=IB SPL-D2, gwg=DFR GBZ(0.3), wlratggkkq=DFR G3(800.0), wjblnczbbx=DFR I(n)3(800.0), ux=IB LL-DZ(0.3), dudvldgw=DFR I(F)B1, ofqyjicucu=DFR I(F)L3(800.0), fpsyw=DFR I(n)B2, icnumkq=IB SPL-L3(800.0), ddt=DFR I(F)L2, qnk=DFR GZ(0.3), iwwsqj=DFR I(ne)L1, rwn=DFR G3(800.0), dzrvez=IB LL-D2, dhedegn=DFR I(ne)3(800.0), qgtq=IB LL-DZ(0.3), vuah=BM25(k1=1.2,b=0.75), rmgee=IB SPL-D1, cmbsxisaoq=DFR I(n)1, ojqmljgk=DFR GLZ(0.3), pwlqcputi=IB LL-LZ(0.3), tkn=DFR I(F)L1, vsuly=DFR I(ne)B3(800.0), ffkgrdgml=DFR I(ne)L2, laqcsujs=DFR I(ne)L3(800.0), klj=DFR I(ne)L1, avt=DFR GL3(800.0), ov=DFR GLZ(0.3), ldhubrkmnz=DFR GBZ(0.3), xhqxejkvg=DFR I(ne)L2, crm=DFR I(F)LZ(0.3), kuaek=IB LL-L1, nualq=DFR I(n)Z(0.3), chxxsm=IB LL-LZ(0.3), alenuqs=IB SPL-LZ(0.3), volkl=DFR I(ne)1, yllehtvud=DFR I(n)Z(0.3), ucbfaff=DFR I(ne)L1, wr=DFR I(F)3(800.0), zhuspkn=DFR I(F)B3(800.0), veyqownn=IB SPL-LZ(0.3), hjl=IB SPL-L2, qab=DFR I(F)BZ(0.3), bhshgwr=DFR G3(800.0), cea=LM Jelinek-Mercer(0.700000), rclngh=DFR I(n)3(800.0), zzzyuq=DFR I(n)L3(800.0), kex=DFR I(F)LZ(0.3), vzotyabzdp=DFR I(n)L3(800.0), nnfzvlvbwl=DFR I(n)B2, ameowxxwj=DFR G1, whwyos=DFR I(ne)L1, ioqzmp=DFR I(F)L3(800.0), oyrfmcw=DFR G3(800.0), pykpq=IB LL-DZ(0.3), w=IB LL-L3(800.0), rz=DFR I(n)LZ(0.3), wgafci=IB LL-D2, fvitvty=DFR I(F)Z(0.3), pwggjojvg=IB LL-L2, qccbpkb=IB SPL-L2, hcsqfilvy=DFR I(F)B3(800.0), bqmnacyo=DFR GB2, tje=DFR I(n)B2, qosrx=DFR I(F)3(800.0), ohxrdhjxu=IB LL-D1, tg=DFR I(n)B3(800.0), shphsysyfj=DFR I(n)Z(0.3), ohtwzkjw=IB SPL-L3(800.0), sklldr=DFR GB1, sycvg=DFR I(F)BZ(0.3), vytrixapl=DFR I(ne)LZ(0.3), pyyrz=DFR G1, msnlbm=DFR I(n)L3(800.0), lfssfqw=BM25(k1=1.2,b=0.75), tqpedc=IB LL-DZ(0.3), fwqta=DFR I(ne)LZ(0.3), esyxenyop=DFR I(ne)1, lee=DFR I(ne)L2, uicls=IB SPL-L2, mpggh=IB SPL-LZ(0.3), hynybdmd=DFR I(n)B3(800.0), zsvpr=DFR I(F)3(800.0), pfc=IB LL-L1, rnnuymshyc=DFR GZ(0.3), elokajxsc=DFR I(ne)2, ewtobwy=DFR I(ne)LZ(0.3), gdjlie=IB SPL-D2, ndddezlbka=IB LL-L1, hloermkbtm=IB LL-L1, gobooqpnp=IB SPL-LZ(0.3), ecqen=DFR I(n)1, jgpsqze=DFR I(n)3(800.0), hczbbkhp=DFR I(n)LZ(0.3), aplln=DFR I(F)2, ojobwzimd=DefaultSimilarity, nmc=DFR I(ne)B1, qgdp=IB LL-D3(800.0), fcgvtrz=DFR I(ne)L2, brvoncd=DFR I(n)L1, vdhyfvfms=DFR I(F)B3(800.0), kio=DefaultSimilarity, wzlvaosa=DFR I(ne)3(800.0), oh=DFR G3(800.0), dgqkx=DFR I(F)3(800.0), ugeda=DFR I(n)LZ(0.3), bcpifeizl=IB SPL-L1, zwoesuq=DFR I(n)3(800.0), mduutge=DFR GLZ(0.3), iml=IB SPL-DZ(0.3), wcn=DFR I(F)L3(800.0), ijiypdmby=DFR I(ne)BZ(0.3), qeiu=DFR G1, spv=IB LL-D2, jnlbrfisek=DFR I(F)B2, izrcrijoiy=DFR GL2, yobpxsz=DefaultSimilarity, prpvlrmzhj=DFR I(F)B3(800.0), ihqdt=DFR GL2, tznirvjvli=DFR G2, hkcubnetkn=DFR I(n)L2, xamd=DFR I(n)1, qmaxwl=IB SPL-DZ(0.3), gaadnynu=DFR I(ne)L1, caktyjgh=IB LL-DZ(0.3), vg=IB SPL-LZ(0.3), hsvuu=DFR I(n)B3(800.0), xer=DFR I(n)L3(800.0), gpt=DFR GLZ(0.3), opwyakx=DFR I(F)B3(800.0), uk=DFR I(F)BZ(0.3), xrfaigb=DFR I(n)L3(800.0), hnoddp=DFR GL1, oeziypdc=DFR I(n)BZ(0.3), vfkurnd=IB SPL-L3(800.0), azjncgxp=IB SPL-LZ(0.3), gsa=DFR GB3(800.0), txubq=DFR I(n)3(800.0), ejhsyptp=IB SPL-D2, un=DFR I(ne)3(800.0), emy=DFR I(ne)B3(800.0), bcmehzhspp=IB SPL-L2, ghioonhqrz=DFR I(F)B3(800.0), onyv=DFR I(F)Z(0.3), nhgdxx=IB SPL-LZ(0.3), bsgoky=DFR I(ne)3(800.0), oaszbjxeru=DFR I(n)B3(800.0), lkpa=DFR I(ne)1, fxe=DFR I(n)B3(800.0), afscpy=DFR GZ(0.3), stbzyv=IB SPL-L1, jqgiz=DFR G3(800.0), lmsxwfvgb=DFR I(ne)L2, me=DFR G1, ofgiri=DFR I(n)LZ(0.3), icnkymbv=IB SPL-D2, ylhdyzhta=IB SPL-D2, cy=IB SPL-D3(800.0), uzdfcy=DFR I(ne)BZ(0.3), eeqd=DFR I(F)B1, ppxl=DFR I(n)B1, jcqjzukkh=DFR I(n)BZ(0.3), spr=DFR I(ne)B3(800.0), mnzdx=DefaultSimilarity, tvdpmxha=IB SPL-D3(800.0), lnwb=DFR I(ne)3(800.0), znkztvqfo=IB SPL-D2, zyd=LM Jelinek-Mercer(0.100000), mnqatxbah=DFR I(n)1, ktyp=IB SPL-DZ(0.3), mjxa=DFR I(n)L1, ncci=IB LL-L2, srwikf=DFR I(ne)B1, j=IB SPL-L1, tfv=DFR I(ne)L2, ekbpilaca=DFR I(n)B1, ujz=DFR I(F)B3(800.0), irfa=DFR I(F)1, fd=DFR I(n)L1, qtknqortsd=LM Jelinek-Mercer(0.100000), xohworfsr=DFR GB2, nvrcsawjs=IB LL-DZ(0.3), qtzqcypvi=IB LL-LZ(0.3), acmg=DFR I(F)1, ugt=DFR I(F)BZ(0.3), vevvweykis=DFR I(ne)L1, d=DFR I(n)2, zubamgz=DFR I(ne)L3(800.0), avhbrb=DFR I(ne)LZ(0.3), cnrhkyt=DFR I(ne)B3(800.0), uednzgrrl=DefaultSimilarity, an=DFR G3(800.0), zww=IB SPL-L2, zhzc=DFR I(F)2, nfsuwlh=DefaultSimilarity, acbte=IB LL-L1, mgnuletja=DFR GB3(800.0), ejwzziosop=DFR I(ne)L2, loo=DFR GB2, pjy=DFR GLZ(0.3), fa=DFR I(F)3(800.0), fuxepyo=IB LL-D3(800.0), abmnk=IB LL-D2, latymg=DFR I(ne)2, jauw=IB SPL-D2, ry=DFR I(ne)B2, pfwomgv=DFR G2, zfnkvcc=IB LL-L2, ueioznhxg=IB LL-D2, f_POSITIONS_AND_OFFSETS_AND_PAYLOADS=IB LL-D3(800.0), wclxcva=IB SPL-L1, rbfeouhtc=BM25(k1=1.2,b=0.75), ildnbqeew=IB LL-L3(800.0), qplkgua=LM Jelinek-Mercer(0.100000), wzqy=DFR I(F)L1, qvap=DFR G1, pwno=DFR I(n)1, oligjyxzu=DFR I(F)B1, ulauxrra=DFR I(F)L2, sozfy=DFR G3(800.0), fhen=DFR I(ne)BZ(0.3), kho=DFR I(n)L1, ueh=DFR G1, ffshyi=DFR I(F)3(800.0), ezhouqr=DFR GB1, swjijwntu=IB SPL-LZ(0.3), litud=DFR I(n)L1, twgva=DFR I(ne)BZ(0.3), wgpo=DFR I(n)Z(0.3), zgeqs=IB SPL-L3(800.0), gasdpqs=IB SPL-D2, hhd=IB LL-D1, paazi=LM Jelinek-Mercer(0.700000), dstdlgmena=DFR I(n)B3(800.0), ftsifieo=DFR GB3(800.0), ibc=DFR G2, qerd=DFR I(n)L1, mfy=DFR I(ne)L2, vamgwoe=DFR I(F)1, gvbc=DFR I(n)LZ(0.3), awk=IB SPL-LZ(0.3), cjmndfjc=IB SPL-LZ(0.3), vetz=IB LL-L1, k=LM Jelinek-Mercer(0.700000), uccp=DFR I(n)L2, ncxrwldvgn=DFR GB3(800.0), ohpwqszmd=DFR I(F)2, hlmlcngveh=DFR I(ne)B1, olfjqwkt=DFR I(F)Z(0.3), dtddy=DFR I(ne)LZ(0.3), cpy=DFR I(n)LZ(0.3), gewrpbeyf=DFR GL3(800.0), huhejdhqzf=DFR GBZ(0.3), nmlpezj=IB SPL-L3(800.0), ndqfrfp=IB SPL-LZ(0.3), betfv=IB LL-L1, hxovbgna=DFR I(ne)L1, vu=DFR GBZ(0.3), tswgwn=IB SPL-D3(800.0), wblnw=IB SPL-L1, pibzavds=DFR I(F)2, ctcxnirs=DFR I(F)L3(800.0), oxmaogdwy=LM Jelinek-Mercer(0.100000), qxzomaxb=DFR I(ne)L3(800.0), yshbt=DFR I(ne)B1, yj=DFR I(ne)B3(800.0), uklbhvcj=LM Jelinek-Mercer(0.100000), lis=DFR GB1, kpopjpkv=DFR I(F)2, hgphxozjlb=DFR I(n)B1, wxdirlk=IB LL-L3(800.0), askml=DFR G1, crhhguzbt=DFR I(n)L3(800.0), bmjhpv=DFR I(ne)B3(800.0), wfirsqyyr=DFR I(ne)L2, xdntwcm=DFR I(n)L2, lubyu=DFR GB3(800.0), joofcw=DFR I(ne)B1, gjfeaki=DFR G3(800.0), cao=DFR I(ne)B3(800.0), sv=DFR I(n)L2, gafvmee=DFR I(ne)B3(800.0), tkx=DFR I(ne)LZ(0.3), saax=DFR I(ne)L1, tb=DFR I(F)L3(800.0), oymtgfue=DFR GB2, ezwhgsz=DFR I(F)L2, olyimoihi=DFR GL1, hyzmk=LM Jelinek-Mercer(0.700000), dcslxy=DFR GLZ(0.3), aoyagvgehf=DFR I(n)B3(800.0), dqserxf=DFR I(F)B3(800.0), cozzb=DFR I(ne)Z(0.3), uee=DFR I(F)B3(800.0), oqbowerip=DFR I(ne)L2, ohnju=DFR I(F)L2, yhxsng=IB SPL-LZ(0.3), ivdu=DFR GZ(0.3), cwli=DFR I(ne)3(800.0), ivyxy=DFR I(ne)L1, rvfzr=DFR I(F)BZ(0.3), ncottxtml=IB LL-L3(800.0), edlazknr=DFR I(n)BZ(0.3), wjj=LM Jelinek-Mercer(0.700000), eijvtqzn=DFR I(ne)L1, lwbidylg=DFR I(F)BZ(0.3), ckjxtpmai=IB LL-L3(800.0), eywwvu=LM Jelinek-Mercer(0.100000), imcrpl=LM Jelinek-Mercer(0.100000), zpznbnmn=DFR I(n)B2, xtw=DFR I(n)L2, qkx=IB SPL-L3(800.0), rxnoky=DFR I(F)B1, ed=DFR GBZ(0.3), obz=DFR I(F)B1, ahhgsnbod=DFR GL2, buuqcldwkw=DFR I(n)BZ(0.3), x=DFR G1, vnacszuln=IB SPL-L3(800.0), vafrixvaww=DefaultSimilarity, sw=IB SPL-DZ(0.3), mscmngsv=IB SPL-DZ(0.3), orwyxmt=IB LL-D3(800.0), wlnn=DefaultSimilarity, ixl=DFR I(n)L3(800.0), bronot=IB SPL-D1, pkckagk=DFR I(F)L3(800.0), olyokmhmr=DFR I(n)B1, rce=DFR I(F)L2, ghyx=DFR I(ne)B1, wquurl=DFR I(n)B1, kycdr=DFR I(n)Z(0.3), bgzczr=IB SPL-L1, hl=DFR I(n)B1, otxfz=DFR I(n)B3(800.0), aoessp=DFR I(F)L1, wuaeai=IB SPL-L2, wbp=IB LL-L1, lwxsfa=DFR I(n)Z(0.3), hjfa=DFR I(n)L3(800.0), pfjkryxca=DFR I(n)2, ucbj=BM25(k1=1.2,b=0.75), feojrqdgfn=DFR I(ne)1, zdjwsdsmhq=DFR I(ne)3(800.0), uxsldzn=DFR I(n)3(800.0), pcbunq=DFR I(F)B1, fxsytlrxsr=DefaultSimilarity, nccu=DFR I(F)B1, bgetywnus=DFR I(ne)L3(800.0), jvss=DFR I(F)L2, syl=DFR I(F)B2, icicbj=DFR I(ne)2, yr=DFR I(n)L3(800.0), ttw=DFR I(F)2, ds=DFR I(n)B2, bbmmqcvx=DFR I(F)Z(0.3), htfh=DFR I(F)BZ(0.3), qfidkdhml=DFR I(ne)Z(0.3), izdrsvebu=IB SPL-L1, nyqfyza=DFR I(ne)B1, gzblvhtfuz=DFR GL2, zibtrexpaa=DFR I(n)3(800.0), ikw=IB LL-L1, gresucgxs=DFR I(ne)L1, drmrj=DFR I(n)B2, xmfazshcob=IB LL-L1, wxxesah=DFR I(ne)B1, otbhlltrob=DFR I(n)3(800.0), rvmaegj=DFR I(ne)B1, twh=DFR I(ne)LZ(0.3), qfedpqz=DFR I(F)B3(800.0), htua=DFR I(ne)1, iqqqtsp=DFR I(ne)B3(800.0), pvsx=DFR GLZ(0.3), ngqtd=DFR I(ne)L2, ptwtsmupv=DFR I(ne)B1, pnmhqi=DFR I(n)Z(0.3), aub=DFR I(F)LZ(0.3), efdi=IB SPL-L2, rr=DFR G1, labjaerax=DFR I(n)2, iy=IB SPL-D1, tgmdn=IB LL-D1, tttyfq=DFR I(ne)1, nlpwcyrqi=LM Jelinek-Mercer(0.700000), vvojw=DFR I(n)2, efxxg=DFR I(F)LZ(0.3), vr=DFR I(n)B1, mrfevbxhgk=DFR GB2, ocfmg=DFR I(F)L2, diwb=DFR GB2, hlwfyczqlb=DFR GZ(0.3), pdwmww=IB SPL-L1, tm=DFR I(ne)LZ(0.3), qxhezjdn=DFR I(n)L1, dfzozhz=DFR I(n)L3(800.0), ahtut=DFR I(n)2, vsixoxe=DFR I(n)2, kjexrmrjsz=DFR I(n)B1, f_POSITIONS_AND_PAYLOADS=IB LL-L1, iwvd=IB LL-D1, dhqxdng=DFR I(F)BZ(0.3), nuof=DFR I(ne)Z(0.3), lqoptp=DFR I(ne)L3(800.0), hrsvcutr=DFR I(n)B3(800.0), umudjp=DFR I(F)3(800.0), mxlbg=DFR I(F)LZ(0.3), ndlcasb=DFR I(F)B1, dkc=DFR I(n)B1, rnkhgddzw=DFR I(n)Z(0.3), texqlj=DFR I(n)LZ(0.3), tmmte=DFR I(ne)3(800.0), hmhwfif=DFR GL3(800.0), uxt=LM Jelinek-Mercer(0.100000), agjy=DFR GL1, cgjjwales=DFR GB2, tzgk=DFR I(F)B2, gayg=DFR I(n)1, vqhlbls=IB SPL-L2, uktejtws=DFR GL1, dcbyzjl=DFR I(F)BZ(0.3), ncpgqmcp=DFR I(n)B2, npzqmbxjr=DFR I(F)B2, ndcjzm=DFR G1, usk=DFR I(ne)3(800.0), zhalyy=IB LL-L2, ehnssh=DFR I(F)B1, aaxaoevj=DFR I(n)Z(0.3), ltvtr=DFR I(ne)L2, tttt=DFR GB2, bpzojuiqox=IB SPL-D2, iwphxin=IB LL-L1, qhkn=DFR I(n)L1, nn=DFR I(n)B1, xux=IB LL-L3(800.0), melwgly=DFR I(F)B3(800.0), pxukyt=LM Jelinek-Mercer(0.700000), aqxub=DFR I(ne)3(800.0), djobew=IB SPL-D2, tmock=DFR GZ(0.3), rnr=DFR I(F)LZ(0.3), kftm=IB SPL-D1, gfsxzd=DFR I(F)LZ(0.3), czlb=DefaultSimilarity, ndihrdh=IB LL-L1, tkfilmhj=DFR GBZ(0.3), ftmry=DFR I(F)LZ(0.3), jmzju=DFR GL2, izqjaag=DFR GBZ(0.3), nqtqizs=IB SPL-L3(800.0), qdb=DFR GBZ(0.3), eaa=DFR GL1, bvgrqr=DFR I(n)B1, fkjosorsvx=DFR GL1, zpsc=IB SPL-L1, fkjogqiv=DFR I(n)L3(800.0), ntt=IB SPL-L3(800.0), wlccxy=IB SPL-L2, naiugt=DFR I(F)Z(0.3), pjxeb=DFR I(n)Z(0.3), rch=DFR I(n)B3(800.0), wcsaa=DFR GBZ(0.3), ijku=DFR I(n)Z(0.3), mgyiac=DFR I(ne)L2, emehv=DFR I(F)L1, mdznodaqxc=LM Jelinek-Mercer(0.100000), oeblcw=DFR G1, jcwyja=BM25(k1=1.2,b=0.75), xnfwjt=DFR I(ne)BZ(0.3), exgxacn=DFR I(ne)2, yvq=DFR I(F)1, wuvthzsx=IB LL-L3(800.0), glkodior=DFR I(F)2, xrg=IB LL-D2, qshwxcaevt=IB LL-D3(800.0), jke=DFR I(n)1, pjmp=DFR I(ne)B3(800.0), lrtgx=DFR I(F)L1, rxlwqmvdp=DFR GBZ(0.3), zixedff=DFR I(n)1, kz=DFR I(F)2, uhkpdz=DFR I(F)L3(800.0), lnxitu=IB LL-D1, prmstuhj=DFR I(ne)B3(800.0), heod=IB SPL-D3(800.0), wbpirumdbs=DFR I(n)B2, kghm=IB SPL-D2, mcoljtodhj=DFR I(F)1, snspy=DFR I(ne)3(800.0), giwkgb=DFR I(n)L2, wryphgit=DFR I(F)1, nkxyanxe=DFR I(n)LZ(0.3), gfs=DFR I(F)Z(0.3), nguukarbl=DFR G3(800.0), kqadxp=DFR I(ne)B2, ferxrsgo=DFR I(F)BZ(0.3), vworb=DFR I(F)L3(800.0), fe=DFR GL3(800.0), bokjovbe=IB SPL-L1, ac=DFR GZ(0.3), wiykmfsvj=DFR I(ne)Z(0.3), pvextpbly=DFR I(F)B2, gdg=DFR I(n)L1, drzuhp=DFR I(n)Z(0.3), hdwtlsewcz=DFR I(F)B3(800.0), mbdxmkdka=DFR GLZ(0.3), ovlqedimjg=DFR I(ne)L2, zts=DFR I(n)BZ(0.3), drtzbdmis=DFR I(F)B1, thomomrhmy=IB SPL-L1, pdwdryjkag=IB LL-L1, tdimpuitmr=DFR I(n)L2, mkzavkj=IB LL-D1, kc=DFR I(n)B1, cfwxl=DFR I(ne)L2, ddfby=IB SPL-D2, wmfipsyxze=IB LL-D3(800.0), kbov=DFR GZ(0.3), hjhoodllg=DFR I(ne)BZ(0.3), vdsozkeig=DFR I(ne)2, qdid=DFR I(F)BZ(0.3), pflbldvza=DFR I(ne)L1, shgkaxsc=DFR G1, wcv=DFR I(n)L1, f_NONE=DFR I(n)B3(800.0), bqeauyrrx=DFR GL1, hz=DFR GZ(0.3), zgri=IB LL-D1, udg=IB SPL-DZ(0.3), ccbhejcc=IB LL-L3(800.0), xtvtiku=IB LL-DZ(0.3), upwoi=DFR I(F)LZ(0.3), mralavcq=DFR I(n)Z(0.3), hhnmheqzbs=IB SPL-L3(800.0), ptlao=IB LL-D3(800.0), aq=DFR GB2, kqoydatb=DFR I(ne)B3(800.0), oklwo=DFR I(ne)1, mev=DFR GZ(0.3), kvxw=DFR GL2, kmwpgy=DFR I(ne)B2, mvdxmar=DFR I(n)L3(800.0), rgxych=IB SPL-D2, tqpmjrtid=IB LL-L3(800.0), azvlyz=IB SPL-D1, lrskvvu=DFR I(n)Z(0.3), irdavmwd=DFR I(F)LZ(0.3), zfypavqplt=DFR I(n)Z(0.3), xupjwtdw=IB LL-L1, gsw=DFR I(n)Z(0.3), bjpeptb=DFR I(F)B1, wd=DFR I(ne)L1, hllugbh=DFR I(ne)L2, sloojzjd=LM Jelinek-Mercer(0.100000), nyeo=DFR GB3(800.0), ve=DFR I(n)B2, nshzcgo=DFR I(ne)2, kltvsn=DFR I(ne)Z(0.3), lipb=DFR I(F)3(800.0), rshkqizvw=DFR I(ne)LZ(0.3), zszwfacfj=DFR GB1, dvteio=DFR I(F)L1, zwfpzkpwpd=DFR I(ne)B3(800.0), rgkkq=IB SPL-D1, asjtzlacs=IB SPL-LZ(0.3), fwvvbqvmj=IB SPL-D2, rifdiemx=DFR I(n)3(800.0), sljade=DFR I(n)B3(800.0), eigyty=DFR G3(800.0), zwdgn=BM25(k1=1.2,b=0.75), uquaosuqb=DFR I(F)L3(800.0), iaycozsjk=DFR I(n)L3(800.0), jp=IB SPL-L1, pyrppu=DFR I(n)2, ammahigh=DFR I(ne)L3(800.0)}, locale=es_GT, timezone=America/Campo_Grande
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=327301928,total=425721856
   [junit4]   2> NOTE: All tests run in this JVM: [TestVarGapFixedIntervalPostingsFormat, TestSimpleTextTermVectorsFormat]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextTermVectorsFormat -Dtests.seed=4AF0D6F4E5896BB7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_GT -Dtests.timezone=America/Campo_Grande -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestSimpleTextTermVectorsFormat (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4AF0D6F4E5896BB7]:0)
   [junit4] Completed [15/21] on J0 in 7223.27s, 22 tests, 2 errors <<< FAILURES!

[...truncated 84 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:734: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:671: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:470: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2243: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1447: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1002: There were test failures: 21 suites, 696 tests, 1 suite-level error, 1 error, 53 ignored

Total time: 228 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #911
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 747722 bytes
Compression is 0.0%
Took 0.51 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure